Shamba-MedCare Prompt Engineering

title: Shamba-MedCare Prompt Engineering

published: true

description: Building Shamba-MedCare - an AI-powered plant disease diagnosis tool for farmers. The problem, the failed approaches, and what finally worked.

tags: ai, machinelearning, promptengineering, webdev

series: Building Shamba-MedCare

Some background context:

I am building a simple plant disease diagnosis solution using AI, inspired by my farming background and advancements in intelligent technological tools

You can check out the Shamba-MedCare App here. Sorry for testing, you'll have to use your own api keys until the public launch is available. The keys are stored in the browser's local storage, so they are private

For context here, whenever you read LLM(Large Language Model), I mostly Claude. I like to use LLM since it's generic, and this solution can be fitted to any LLM.

I played around with several prompts in order to nail the best results. This is how I transformed my prompt engineering journey with Shamba-MedCare:

My first prompt to LLM Vision was embarrassingly naive:

"What disease does this plant have?"

The response was a 2,000-word essay about plant pathology in general. Helpful for a textbook. Useless for a farmer with a dying tomato plant. Getting AI to return structured, actionable, budget-aware diagnoses took iteration. Here's what I learned.

The Architecture

Two prompts matter: the system prompt (who LLM(e.g. claude pretends to be) and the analysis prompt (what to do with this specific image).

System Prompt: Creating "Shamba"

Prompts work better with a persona. I created Shamba persona, an agricultural pathologist who:

The key line: "Always include at least one FREE/traditional treatment."

Without that explicit instruction, the LLM defaulted to commercial products. Helpful for a suburban gardener. Useless for a farmer who can't afford a $15 fungicide.

Failure #1: The JSON Nightmare

My first structured attempt asked LLM to return JSON, which it did pretty well. Wrapped in markdown code fences. With helpful commentary before and after.

{ "disease": "Early Blight" }

My parser choked. The fix was explicit:

Still failed 10% of the time. So I added backend parsing that:

1. Strips markdown fences if present
2. Extracts JSON from surrounding text
3. Validates against the expected schema

Failure #2: Location Descriptions

For the visual heatmap feature, I needed LLM to describe WHERE damage appeared. My prompt asked for "affected regions."

LLM returned: "The affected area is significant." This was not helpful, I needed the exact coordinates, and I tried out several solutions. This was close to perfect:

Now LLM returns:

That's enough to generate a heatmap overlay for now

Failure #3: Treatment Cost Blindness

Early on, treatments came out randomly ordered. Sometimes the $50 systemic fungicide appeared first. Sometimes, the free wood ash remedy.

The problem: LLM has no inherent understanding of budget constraints. I had to structure it:

The response schema enforced this:

The Plant Part-Specific Prompt Strategy

Different plant parts reveal different problems, and I needed the right prompt to get the right problem with the best remedies. My prompt adapts:

For leaves:

For roots:

This focus improves accuracy dramatically. Asking LLM to look for "anything wrong" produces vague results. Asking it to specifically check for concentric ring patterns in leaf spots? Now we're diagnosing Early Blight.

The Final Prompt Structure

What I'd Do Differently

Start with the output format first. I designed prompts around what I wanted LLM to do. I should have designed around what the farmer needed to see.

The heatmap feature was an afterthought. If I'd planned for it from day one, the location description format would have been baked in, not retrofitted. This is actually a useful feature for farmers if you can imagine an affected plant, there are heatmaps on the heavily affected areas

Test with bad photos early. My development photos were well-lit, centered, single-issue plants. Real farmer photos are blurry, shadowy, and show three problems at once. The robustness I needed only emerged after testing with garbage inputs.

Source code on GitHub