HealthCart
An agentic grocery shopping assistant that turns health conditions into a compliant weekly shopping list, checked by an LLM-as-judge, and stress-tested until it wasn't.
I wrote a line of code that would have let a peanut slip past a nut allergy check as “compliant.” Nobody told me to fix it. I found it by rereading my own prompt a second time, the way you'd reread a text before sending it. That single line is most of why this project exists.
HealthCart is a working prototype, no company commissioned it: a 5-node LangGraph pipeline, LiteLLM model routing, Langfuse tracing, and a 20-case eval harness, all running on my machine right now. What follows is the record of building an AI that's supposed to catch AI's mistakes, and everything that went wrong while I built it, including the parts I got wrong myself. Where something below is still a plan and not a shipped thing, I say so.
Verification is the product.
The shopping list is just where you can see it.
129 million Americans live with a chronic condition where diet does most of the work: Type 2 Diabetes, Celiac Disease, Chronic Kidney Disease, Hypertension. The person trained to turn a diagnosis into an actual grocery list, a registered dietitian, charges $70 to $300 a session out of pocket, and Medicare only covers three conditions. Everyone else figures it out alone.
Grocery apps answer this with tags stapled onto products: vegan, gluten-free, keto. None of them start from a person's actual body. And when people ask a general-purpose AI to fill that gap instead, it doesn't just get things wrong. It gets them wrong in ways that can hurt you.
How do you make an AI-derived food recommendation trustworthy before someone acts on it, without bringing back the $70 to $300 barrier the AI was supposed to route around? That's a question about architecture, not filters. It's also why the shopping list itself isn't the interesting part of this project. The second model call that checks the first one's work is, and so is everything I got wrong while building that.
A 6-node LangGraph pipeline
Health profile goes in, a validated shopping list comes out. Three pieces of infrastructure make that possible, and none of them think for themselves.
A shared, typed state object passed from node to node. The graph runs in a straight line today, but building it as a graph (not five function calls chained together) means I can add a branch later, like routing an uncertain judgment to a real person, without touching anything else.
litellm.completion(model="groq/llama-3.1-8b-instant", ...) works the same across 100+ providers. HAIKU and SONNET are just string constants in my code, not a provider-specific SDK I have to swap out by hand.
Every node wraps in @observe(), which builds a trace tree I can actually open and read. It also takes custom scores, which is how eval results and human feedback both end up attached to a specific run.
Checking my own work turned up three real bugs
A 20-case eval harness covers single and combined conditions, plus one safety-critical case: Peanut and Tree Nut Allergy, with the bar raised to 85 percent compliance instead of the usual 70 to 80. A false negative there isn't a minor annoyance. It's someone in the ER.
The compliance validator's own prompt, which I wrote, said this:
- Mark non-compliant ONLY if it directly contradicts a constraint - When in doubt, mark compliant: true.
That's a fine default for most constraints. You don't want spinach flagged for no reason. It's the wrong default for allergies specifically. An eval that reports 90 percent average compliance tells you almost nothing if the judge's own instructions nudge it toward “compliant” on the exact failure that matters most.
I split the default. General constraints stay lenient. Allergen constraints now flip to non-compliant when the judge is unsure, and every item gets tagged allergen_relevant: true/false so that group can be measured on its own.
for item, judgment in zip(recommendations, judgments): validated.append({"passed": bool(judgment.get("compliant", True)), ...})
An 8B model has no guarantee it hands items back in the order it received them. Only a sentence in the prompt asking nicely does. Drop one item or reorder two, and this code quietly pins the wrong verdict to the wrong food. No crash. No warning. Just a mistake you'd never see.
The judge now echoes each item's name back to me, so matching happens by name first, with position as a fallback I actually flag instead of trust blindly. I also flipped the missing-key default from True to False: an incomplete judgment no longer reads as a silent pass.
Building the human-feedback feature meant I needed the SDK's real API, not the one I assumed from the README:
>>> hasattr(Langfuse(public_key='x', secret_key='y'), 'trace') False
eval/runner.py had been calling langfuse.trace(...) since day one. That method does not exist on the installed v3 SDK, which swapped it out for start_as_current_span() and score_current_trace(). Every eval run's scores had been quietly failing to log, caught by a broad except Exception nobody was looking at closely enough to catch. The harness had been lying about itself, and nothing would have told me unless I went and checked by hand.
Rewired it to the real v3 API. Confirmed working: a real trace ID now comes back from every eval run.
A guess I said out loud with more confidence than I'd earned
I wired in USDA FoodData Central grounding after confirming Instacart's Developer Platform is closed to new applications right now (and takes 30 to 40 days to approve even when it's open). The first run gave me a strange result: 0 percent allergen compliance, and only 5 of 16 items grounded against USDA at all.
“USDA's plain, scientific food names don't match the model's marketing-style item names, like ‘Wild-Caught Salmon.’” Sounded right. Was wrong. Spinach, Lentils, and Quinoa failed to ground too, and there is no naming story that explains that.
So instead of writing the guess into the docs, I tested it.
>>> requests.get(FDC_SEARCH_URL, params={"query": "Spinach", "api_key": "DEMO_KEY", ...})
{"error": {"code": "OVER_RATE_LIMIT", "message": "You have exceeded your rate limit..."}}The free, shared DEMO_KEYrate-limits hard, and one 16-item list fires 16 sequential USDA calls. Most of that run was hitting HTTP 429, not “no match found.” My own except Exception: return Nonehad been flattening two totally different situations, “USDA has nothing on this” and “I couldn't even check,” into one signal.
search_food() now returns a real status: grounded, no_match, or lookup_failed. Not one collapsed boolean pretending to know something it doesn't.
Collapsing those three into one “ungrounded” state would have been the exact mistake this whole project argues against: a system that sounds more confident than it has any right to be.
The next thing worth building isn't a new feature
create_score(trace_id=...) straight onto the exact run that produced it.