Last Updated: October 12, 2024.

Quick Links

Product Landing:  https://designcrit.ai
Figma Plugin:  Figma.com/community/plugin/designcrit-ai

Background

I’ve been interested in the intersection of design & machine learning for quite a few years now. So, what better way to learn more about it then to throw myself into creating a new product that tries to solve problems for my fellow designers.

I built something similar before
A few years ago, when I was still running Adjust Creative, I built a small AI tool prototype for homepage design critiques using IBM Watson. At the time, NLMs were in a pretty new state, so I couldn’t just use them out of the box, I had to train the model myself with a lot of screenshot images of websites and then give each image a weighting scale for categories I wanted the model to grade the design. These categories were design fundamentals like color theory, fonts, and layout coupled with more product squeeze page metrics like call-to-action, clarity of communication, value statement, etc.

AI has come a long way since
It was a lot of training for an arguably mediocre tool. But, it taught me about the power of AI, how to continuously train models, and we used it as a lead generator to capture some folks looking for better design on their websites.

Why build this?

This specific question happened in a 1:1 with a designer recently:

"What if there were a tool where I could get this quick feedback without bothering you, that way I could just talk to you about larger problems?"

I didn’t feel bothered. I’m happy to give other folks feedback and help when I can. But, she posed an interesting question. This person was very much looking for quick and direct feedback that showed me an opportunity to learn, help, and make cool things.

Remote designers need quick feedback
I work as a designer, I’ve been hybrid or remote for years. I miss the office sometimes—usually because I could lean over to a colleague and get quick feedback about whatever I’m working on.

Less experienced designers need guidance
A lot of designers who haven’t yet built up skills to make certain decisions quickly based on experience and intuition need the guidance of someone senior at their reach.

Where should I build this?

Figma plugin first
Because my idea was for designers specifically, there’s no better place to build and test the prototype than in a Figma plugin. I have experience building Figma plugins—that’s where designers are—so that’s where the bulk of the prototype development went.

Web UI or other
There could be multiple tiers to this project in the future, including a full featured web dashboard. But, for prototypes and getting something out the door, I only need a Figma plugin and a product landing page.

A couple of key points:

• Figma plugin works directly with Figma files
• Web takes a URL and gives you some general feedback
• Web UI points you to Figma plugin
• Web UI drives you to contact me

What were the biggest challenges?

Being opinionated about the interaction design
Allowing free prompt vs making it UI based prompting. I think for most users, especially ones that aren’t familiar with AI tools yet, the button/settings based version is more useful. Maybe there could be a more power user mode for advanced users.

Google Cloud AI

Why Google Cloud?
I tested OpenAI and Google Cloud with this app. Google Gemini seemed to respond slightly better than OpenAI and more consistently. However, I’m still evaluating this decision. These things are improving an a crazy speed, so they sort of leap frog each other. There’s also a possibility of getting a machine, downloading Meta’s ML models and hosting it. But, that’s a much bigger level of effort.

Google Cloud AI Budget limits
There’s no budget limit on Google Cloud AI—which means you could accidentally go bankrupt if it gets wildly popular in a short amount of time. For the beta, I had to write in a request-per-day limit on the ruby app to make sure things don’t get out of hand for the testing phase…

Markdown conversion
The API responds with markdown and needs to be converted.

Figma Dev

Uint8Array to Base64
Converting large arrays needed to be broke into chunks for performance reasons

Figma Plugin UI vs Backend
Was a little tricky realizing a lot of data needs to be processed on Node backend vs UI window and how that data is sent back and forth. So, it meant creating a very event based app.

A lot of changes that require screen updates need to post messages to the Node backend to make the decisions, then that backend needs to send a message back to the Plugin UI to update.

Ruby on Rails

Google Gemini tokens
The API responds with tokens while it’s processing and streaming data to you, so you need to concat it afterwards or stream chunks to the UI.

Design Goals

Easy to use user interface with point-and-click prompt.
Needs to have modern and cute brand voice to offset the “AI of Doom” vibe that a lot of folks get from fresh technologies.

Building The Right Prompts

Since the prototype of this is going to be utilizing generative AI, we need to be able to prompt it correctly. Giving a very specific prompt in the right way is a very detailed task.

What kind of feedback will this give to people?
The way I set up the plugin is that the prompt has multiple levels of instructions that then get compiled and sent to the server. This will likely move to the server as well so that ML can be used to surface recommendations for common interactions.

At first, it was just a “hey give me feedback about this design” kinda prompt. Then it evolved to a per design fundamental prompt. This gives it more detailed information about specific parts of a design, such as:

Information Hierarchy
Navigation
Usability
Fonts
Color
Scanability
Graphic Design

Who is the one giving the feedback?
After using it a few times, I realized, I need to set a different audience to get the right kind of feedback. So, I added a persona portion that allows folks to choose a couple of different personas. It really does make a big difference in the type of feedback that is returned.

Senior Designer
Marketer
Hiring Manager

Further context for what type of art i’ve selected
This got pretty far, but it still didn’t know the difference between an app screen vs a website vs just a drawing. Enter the Design Context:

UX
Graphic
Poster
Icon

Specific rules
The feedback was good, but it needs honed over time. Some of the things that made it into the MVP are “how to respond” type context, such as:

Only Respond in english—it was responding sometimes in partly Japanese???
When giving negative feedback, try to give a “why"
Start the response with overall feedback, then bullet point the specifics

This part will change the most until the testers and I get it to be as valuable as it can with this level of generative AI. Then, from that point, I’ll start training the models with expert data so it has better context and understanding per each topic.

MVP Launch

Let’s go!
As soon as I had all the necessary tech and design in place and it functioned as intended—I rolled out the product publicly. Like, literally within 2 hours.

I got the domain DesignCrit.ai after running through about 10–20 name ideas + generating many more with ChatGPT.

Built a pretty quick product landing page so that there’s a bit of face for this application and a place to store documentation, FAQs, and testimonials when they are ready.

User Testing

With the plugin in the Figma plugin community, I asked UX designers on LinkedIn to help me test it. The best way to test features is with an active community, so I started a Discord server for us to throw ideas around in to improve the plugin.

There are categories like Design Feedback, Prompt Response Feedback, and Bugs in the beta testing group. In the first two weeks, there are already a handful of new product requests and updates!

If you’re a designer or programmer and would like to get involved in helping iron out the beta of this plugin, feel free to reach out to me to request access to the Discord.

Future Iterations

As of October 2024, I’m currently honing the product and I’m adding to it as I have time after work and on weekends. I would love for this to become useful enough that senior designers could benefit from it just as much as junior designers or even devs trying to get some quality feedback.

My next major dev effort is going to be creating the user authentication and API key requirements so I can better track who’s using the plugin and AI resources.

* Note: When I have more, I’ll edit and update this article.

I also put this article on Medium, but with some code blocks in case it helps someone else out there trying to work on similar things.