Anybody can send a PCB description/schematic into an LLM, with a prompt suggesting it generate an analysis and it will diligently produce a document that perceptually resembles an analysis of that PCB. It will do that approximately 100% of the time.
But making an LLM actually deliver a sound, useful, accurate analysis would be quite an accomplishment! Is that really what you've done? How did you know you got it right? How right did you get it?
To sell an analysis tool, I'd expect to see some kind of comparison against other tooling and techniques. General success rate? False negative rate? False positive rate? How does it do against simple schematics vs large ones? What IC's and components will it recognize and which will it fail to recognize? Does it throw an error if it encounters something it doesn't recognize? When? Do you have testimonials? Examples?
Benchmarking is tricky right now because there aren’t many true “LLM ERC” systems to compare against. You could compare against traditional ERC, but this tool is meant to complement that workflow, not replace it. For this initial MVP, most of the accuracy work has come from collecting real shipped-board schematics (mine and friends’) with known issues and iterating until the tool consistently detected them. A practical way to evaluate it yourself is to upload designs you already know have issues, along with the relevant datasheets, and see how well it picks them up. Additionally, If you have a schematic with known mistakes and are open to sharing it, feel free to reach out to through the "contact us" page. Contributions like that are incredibly helpful, and I’d be happy to provide additional free usage in return.
I’ll also be publishing case studies soon with concrete examples: the original schematics, the tool’s output, what it caught (and what it missed), and comparisons against general-purpose chat LLM responses.
The goal isn’t to replace a designer’s judgment, but to surface potential issues that are easy to miss. Similar to how AI coding tools flag things you still have to evaluate yourself. Ultimately the designer decides what’s valid and what isn’t.
I really appreciate the push for rigor, and I’ll follow up once the case studies are live.
I see this idea as a sort of AI ERC/DRC checker that offers some incredible opportunities. Even if it only catches one small, it could save thousand of dollars down the line.
It's another tool in the toolbox for hardware designers.
Or it could send a design team down thousands of dollars in false positives/false negatives. With zero benchmarks provided, it is very fair to question a product that could have material negative impacts on a hardware team.
I know a brilliant PCB engineer whose first major multimillion dollar R&D corporate design (decades ago) resulted in production of a modular product which couldn't physically plug in with the rest of the system (because of above issues)... I'll send him this link to see if he'll give you feedback, but that's going to be how he'd initially test your AI system (he considers it a humbling lifetime blunder).
Without any PCB design experience, my presumption is that OP's "AI product" is more of just a "fundamentals of circuit board design"[0] and not an all-expansive "how did no human ever catch such a simple multi-dimensional clash"[1]
[0] isolated voltage areas; trace attenuation avoidance; signal protection
[1] the darn thing won't even plug in, because the plug is pin'd-out backwards
But it sounds like in this case the root cause was more of a footprint/layout issue rather than a schematic one. I’m hoping to add footprint-level checks later on, once I can ingest full board files and mechanical data.
Pinouts... there is a reason we try to get all pinouts tested as early as possible, preferably on the first non-form-factor prototype spin if we can. In no event should key pinouts be first assigned or major changes made without a planned spin in the schedule following them....
I built this because I was tired of shipping boards with avoidable mistakes — hopefully it saves you from a re-spin too!
Datasheets themselves are inconsistent and incomplete so I’m wondering how you evaluated the accuracy of the import and what your acceptance criteria is.
I’d really recommend trying it with one of your designs: upload the netlist + a component’s datasheet and ask a specific question about the part in the design. It’s the easiest way to see how well the ingestion works in practice. Would love to hear your feedback after you try it!
I’m not against more automated checkers, I’m very much for automated checkers, but I’m curious how you plan to not repeat the mistakes of the past.
> Of course, Jack. I can understand the schematic from the provided JSON file. It describes an RS485 to TTL Converter Module. > Here is a detailed breakdown of the circuit's design and functionality
...followed by an absolutely reasonable description of the whole board. It was imprecise, but with some guidance (and by putting together my basic skills with Gemini's vast but unreliable knowledge) I was able to figure out a few things I needed to know about the board. Quite impressive.
Also good call on processing EasyEDA schematics. I hadn’t considered that initially, but I’m definitely going to add support for it.
If doing industrial work, than consumer-grade workmanship / LLM-slop is usually unacceptable. Start with the FTDI firmware tool and an isolation chip App-note...
https://www.analog.com/en/products/adm2895e-1.html
Best of luck =3