Testing AI Agents That Use MCP: From Working Tools to Reliable Outcomes

Model Context Protocol servers can make AI agents more useful by connecting them to databases, applications, and business workflows. However, a server that responds correctly to a direct request is not automatically ready for real-world use. Once a language model must choose a tool, build the right arguments, interpret the response, and decide what to do next, reliability becomes a system-level challenge.

A good testing strategy separates layers. Unit tests verify the business logic behind each tool. Protocol tests check discovery, communication, message exchange, and error handling. Conformance tests confirm that schemas match what the tool actually accepts and returns. Security and load tests examine permissions, isolation, prompt injection, timeouts, and capacity. Agent evaluations then test the behavior that software checks cannot capture: whether the model finds the correct tool, uses it properly, and completes the user’s goal.

This matters because an MCP tool is not called by predictable application code. It is called by a probabilistic model that reads tool names, descriptions, schemas, instructions, and conversational context. Two tools may appear similar. A required field may be documented but easy to misunderstand. The agent may answer from memory instead of using a tool, make unnecessary calls, repeat the call, or stop before a multi-step task is finished.

Useful evaluations begin with a clear task and a measurable definition of success. Each attempt is one trial, and several trials are needed because model behavior can vary. Graders score different parts of performance, while traces record the prompts, available tools, arguments, responses, intermediate decisions, timing, and final answer. Importantly, the evaluation should check what actually changed in the environment. If an agent says that a refund was issued, the refund must exist in the system of record.

A strong scorecard covers several dimensions. Tool-selection accuracy measures whether the agent chose the required tool and avoided irrelevant ones. Argument checks catch missing, malformed, or incorrect values. Task-completion scoring verifies that all requested work was finished, including dependent calls in the right order. Final-state checks confirm the real-world result. Other measures can assess whether tool outputs are understandable, whether the agent used too many calls or tokens, and whether performance stays consistent across repeated runs.

Dataset quality matters more than size. Start with the tasks developers already test manually, then add common user requests, support incidents, bug reports, and failed production traces. Every confirmed failure can become a regression case. Synthetic tasks can increase coverage, but they should be reviewed by people who understand the domain and can agree on the expected behavior.

A balanced test suite should include more than successful examples. It should cover requests that require no tool, prompts that could match several similar tools, multi-step workflows, missing information, upstream failures, permission limits, and actions with side effects. High-risk operations such as sending messages, deleting records, charging customers, or changing access should verify clear user intent, exact arguments, and execution count. A reliable agent should neither bypass restrictions nor perform an approved action twice.

Different behaviors need different graders. Deterministic checks work best when there is one correct tool, format, value, call count, or stored result. Trajectory checks can confirm dependencies and step limits without forcing one rigid sequence. Model-based graders are useful for semantic qualities such as completeness, clarity, groundedness, or appropriate refusal, but their rubrics should be narrow and tested against human judgment. One average score can hide serious failures, so thresholds should be set separately for different risk levels.

Capability testing and regression testing should also remain separate. Capability suites should contain difficult tasks that reveal where the agent can improve. Regression suites should protect behaviors the agent already handles reliably. As performance improves, solved cases can move into the regression suite while harder cases take their place. This prevents scores from becoming meaningless and helps teams detect cases where progress on complex tasks causes failures on routine ones.

When an evaluation fails, the trace should guide the diagnosis. First, confirm that the grader is fair. Then inspect tool choice, schema clarity, argument construction, server output, agent instructions, and model capability. A skipped tool may point to a weak description. Bad arguments may reveal confusing field names. A correct call with the wrong result may indicate a server problem. Comparing the same case across models or tool definitions can help isolate the cause.

Finally, evaluations should become part of the development and release process. Version datasets, prompts, schemas, tool descriptions, models, and graders. Run fast regression tests on proposed changes, use broader suites before releases, and block updates that violate critical safety thresholds. After launch, combine automated evaluations with production monitoring, user feedback, transcript review, and controlled experiments. The goal is not a perfect score, but a living measurement system that makes failures visible, improvements comparable, and agent behavior increasingly dependable.

The Best Leadership Lessons I’ve Learned as an Engineering Manager

Engineering management is often described as a technical leadership role. But most of the job is not about code. It is about people: understanding them, giving feedback, handling conflict, building trust, and helping a team do its best work.

Several books have shaped how I think about this: Surrounded by Idiots, Surrounded by Bad Bosses, Radical Candor, and How to Win Friends and Influence People. Here are the lessons I find most useful.

People need different things from you

Surrounded by Idiots describes four broad communication styles using colors. Some people are fast and direct. Some are social and full of ideas. Others are calm, patient, and supportive. Some prefer detail, structure, and clear facts.

The labels are not perfect, and people should not be placed into fixed boxes. Still, the main lesson is valuable: the way you prefer to communicate may not work for everyone.

As an engineering manager, I try to adjust my approach.

A developer who likes detail may need a written plan before starting. A fast-moving engineer may prefer a quick decision and the freedom to act. A quieter team member may need time to think before sharing an opinion.

Good communication is not only about being clear. It is about being clear in a way the other person can understand.

Do not become the bad boss you once had

Surrounded by Bad Bosses shows how leadership problems often come from a mismatch between the manager and the employee. A manager may think they are being helpful, while the employee feels controlled, ignored, or rushed.

This is an important reminder: your intention matters less than the effect your behavior has on the team.

For example, checking progress every few hours may feel like support to you. To the engineer, it may feel like a lack of trust. Giving someone complete freedom may feel empowering, but they may experience it as a lack of direction.

Ask simple questions:

“What support would help you most?”

“Do you want advice, feedback, or space to solve this?”

“Is my level of involvement working for you?”

Strong managers do not assume they know what people need. They ask.

Be kind enough to be honest

The core idea of Radical Candor is simple: care personally and challenge directly.

Feedback should not be cruel, but it should also not be hidden behind vague language. Saying “Everything looks fine” when someone is struggling does not protect them. It prevents them from improving.

Useful feedback is clear, specific, and focused on behavior.

Instead of saying, “You need to communicate better,” try:

“When the release date changed, the rest of the team did not hear about it until the next day. Next time, please post an update as soon as the plan changes.”

It also helps to ask for feedback before giving it. A manager who welcomes criticism makes honesty safer for everyone.

Make people feel seen

How to Win Friends and Influence People teaches a timeless lesson: people respond better when they feel respected and understood.

Learn what matters to each person. Listen without preparing your reply. Give honest appreciation. Admit when you are wrong. Avoid embarrassing people in front of others.

In engineering teams, recognition does not always need to be dramatic. Thank someone for improving a test suite, helping a teammate, writing clear documentation, or raising a risk early.

People are more likely to accept your feedback when they know you also notice their good work.

The main lesson

The best engineering managers are not the ones with all the answers. They are the ones who stay curious about people.

Adapt your communication. Check the effect of your leadership. Give honest feedback with care. Listen closely. Show appreciation. Admit mistakes.

Technical knowledge may help you earn trust at the start. How you treat people determines whether you keep it.

What We Learned About Building MCP Servers That Hold Up in the Real World

 

We recently spent time with two excellent posts on MCP server design: GitHub’s guide to secure, scalable remote MCP servers and Block’s playbook based on building more than 60 of them. Both gave us plenty to think about—and, honestly, they also confirmed a lot of what we’ve been seeing in our own work. 

The biggest takeaway is that a good MCP server is not simply an existing API with a new wrapper. Block makes a strong case for designing from the user’s workflow backwards. Instead of exposing dozens of small endpoints and asking the model to stitch them together, it is usually better to offer a smaller set of well-named, higher-level tools that complete meaningful tasks. Tool names, descriptions, parameters, and outputs all act as instructions to the model, so clarity really matters. Fewer calls, concise responses, and actionable errors generally make the whole interaction more reliable.

That matches our experience. Long chains of tiny tool calls may look flexible on paper, but they add latency, consume context, and create more opportunities for something to go wrong. The best results often come from keeping the interface simple while handling the real complexity behind the scenes. It also helps to design around what language models do well—such as querying clean, structured data—rather than forcing them through unnecessarily complicated plans or oversized payloads. Block’s advice to watch token budgets, truncate or paginate large results, and separate read-only tools from higher-risk write operations is especially practical.

GitHub’s post adds the equally important production perspective: security and scale need to be part of the design from day one. Remote MCP servers can sit between agents and sensitive systems, so authentication alone is not enough. Tokens must be validated carefully, including their intended audience; permissions should follow least privilege; and every database query, cache lookup, and downstream request must stay scoped to the correct user. GitHub also recommends established OAuth patterns, dedicated secret-management services, workload identities, and strong observability instead of custom security shortcuts.

This strongly reflects what we have learned as well. Once an MCP server moves beyond a prototype, the less visible details become the most important ones: rate limits, caching, circuit breakers, secret rotation, structured logs, traces, health checks, and useful alerts. A gateway can centralize many of these concerns and keep them out of the core business logic.

Our shared conclusion: successful MCP servers are workflow-first, security-first, and deliberately boring—in the best possible way. When the tools are easy for the model to understand, safe for users to approve, and straightforward for teams to operate, MCP becomes far more than a demo. It becomes dependable infrastructure.

Leading with Influence, Ownership, and Technical Judgment

A strong engineering leader is not defined by authority, but by influence. That is the enduring lesson of How to Win Friends and Influence People. Dale Carnegie’s core message is simple: people respond best when they feel respected, heard, and valued. In practice, that means listening before prescribing, giving sincere appreciation, avoiding public criticism, and aligning around shared goals instead of ego. In engineering, this is not soft leadership. It is operational leadership.

Coaching starts there. Great leaders do not try to be the smartest person in the room; they create conditions where others can do their best work. That means setting clear expectations, giving direct but respectful feedback, and helping people grow through stretch opportunities. Coach the person, not just the task. Understand what motivates each engineer, where they are stuck, and what good looks like for their level. Hold a high bar, but make it feel achievable.

From a product delivery perspective, technical leadership is about turning ambiguity into momentum. Start with the customer problem, define the desired outcome, and make trade-offs explicit. The team needs clarity on architecture, scope, risks, dependencies, and sequencing. Strong technical delivery is rarely about building the most elegant system; it is about building the right system for the current stage of the product while preserving room to evolve. Good leaders protect engineering quality without losing speed. They know when to invest in foundations and when to ship iteratively.

Ownership is the multiplier. Leaders take responsibility beyond their job description. They do not hide behind unclear requirements, cross-team dependencies, or inherited systems. They surface risks early, make decisions with imperfect information, and stay accountable for outcomes, not just effort. When things go wrong, they do not look for blame; they look for truth, learning, and recovery.

Technical strategy sits above individual projects. To stay ahead, an engineering leader must understand the business, the product roadmap, the system constraints, and the talent on the team. Strategy is choosing what not to do as much as what to do. It is aligning long-term architecture with near-term product value.

In the end, leadership is trust compounded over time: influence people well, coach deliberately, deliver pragmatically, own outcomes fully, and think several moves ahead.

Leading with Clear Judgment

Daniel Kahneman’s Thinking, Fast and Slow explains that people make decisions in two modes. “Fast” thinking is intuitive, automatic, and useful for speed. “Slow” thinking is deliberate, analytical, and necessary for complex judgment. The lesson for leaders is simple: instinct is valuable, but unchecked instinct creates bias. Strong engineering leaders know when to trust pattern recognition and when to slow down, challenge assumptions, and force clearer reasoning.

That principle matters most in coaching. People rarely need a manager who has all the answers; they need one who helps them think better. Coaching means creating enough safety for honest discussion and enough challenge to raise the bar. A good leader does this by clarifying expectations, giving direct feedback, asking better questions, and helping individuals understand not just what to do, but why it matters. Great teams are built when people grow in judgment, ownership, and confidence.

From a product delivery perspective, technical leadership is about turning ambiguity into reliable execution. Start with the customer problem and define measurable outcomes. Then translate those outcomes into architecture, milestones, and trade-offs. Delivering well means slicing scope intelligently, reducing risk early, and making quality non-negotiable through testing, observability, and operational discipline. Technical leaders do not chase elegance for its own sake; they build systems that are maintainable, scalable, and aligned with business value.

Ownership is the thread that connects all of this. Responsibility means not hiding behind role boundaries, incomplete requirements, or other teams. It means raising risks early, making decisions with incomplete information, and staying accountable for outcomes, not just effort. Mature leaders do not ask, “Who caused this?” first. They ask, “What is the reality, what is my part in it, and how do we move forward?”

Being on top of technical strategy requires the same balance of fast and slow thinking. You need fast pattern recognition to spot trends, talent, and architectural drift. You need slow thinking to evaluate long-term bets, platform investments, technical debt, and organizational design. The best engineering managers connect today’s delivery to tomorrow’s capabilities. They create teams that execute now while building the foundation for what comes next. That is leadership: sound judgment, disciplined delivery, and consistent ownership.

Fallacies in Disagreement

In practice, you do not need to memorize every Latin label for bad arguments. Most fallacies fit into a few stereotypical classes.

First are relevance fallacies: arguments that distract from the claim instead of addressing it. This includes ad hominem (“you only think that because…”), appeals to authority or popularity (“the staff engineer said it” or “everyone agrees”), and whataboutism. These usually signal that the conversation has shifted from truth-seeking to status, tribe, or self-protection.

Second are evidence fallacies: weak support presented as strong support. Think hasty generalization, cherry-picking, anecdotal proof, and survivorship bias. In engineering terms, this is drawing a platform-level conclusion from one incident, one customer, or one successful rollout while ignoring the full data set.

Third are causality fallacies: mistakes about why something happened. False cause, post hoc reasoning, slippery slope, and false analogy live here. These show up when people confuse correlation with causation or assume one decision inevitably leads to disaster without intermediate reasoning.

Fourth are framing fallacies: the problem is distorted before it is even debated. Straw man, false dilemma, loaded question, equivocation, and moving the goalposts are common examples. These are especially damaging because they make smart people argue past each other while thinking they are debating the same thing.

What does this mean in real disagreements? A fallacy usually does not mean the person is stupid or malicious. It usually means they are under pressure, attached to an outcome, or reasoning too quickly. Your job is not to “win” by naming the fallacy. Your job is to restore clarity.

A good response sounds like this: “Let’s separate the claim from the person.” “What evidence would change our mind?” “Are we arguing data, causality, or tradeoffs?” “Can we restate the opposing view in a way they would agree with?”

The most useful leadership principle is simple: treat fallacies as failures of argument quality, not failures of character. When disagreements get sharper, lower the temperature and raise the standard of reasoning. That is how teams keep trust while still making hard decisions.

Bending Light: Simulating General Relativity in the Browser with Raw WebGL

As software engineers, we often work within the comfortable constraints of Euclidean geometry: grid layouts, vector positions, and linear interpolations. But what happens when the coordinate system itself is warped?

Recently, I built a real-time visualization of a Schwarzschild black hole using raw WebGL and JavaScript. The goal wasn't just to create a pretty image, but to solve a complex rendering problem: How do you perform ray casting when light rays don't travel in straight lines?

This project explores the intersection of high-performance graphics, general relativity, and orbital mechanics, all running at 60 FPS in a standard web browser.

The Challenge: Non-Euclidean Rendering

Standard 3D rendering (rasterization or standard ray tracing) assumes light travels linearly from a source to the camera. In the vicinity of a black hole, extreme gravity bends spacetime. Light follows geodesics—curves defined by the spacetime metric.

To visualize this, we cannot use standard polygon rasterization. Instead, we must utilize Ray Marching within a Fragment Shader, solving the path of every pixel mathematically in real-time.

Architecture: The "No-Framework" Approach

While libraries like Three.js are excellent for production 3D apps, I chose raw WebGL API for this simulation.

Why?

1. Performance Control: I needed direct control over the GLSL rendering pipeline without overhead.

2. First-Principles Engineering: Understanding the low-level buffer binding and shader compilation ensures we aren't relying on "black box" abstractions for critical math.

3. Portability: The entire engine runs in a single HTML file with zero dependencies.

The Physics Stack

1. Relativistic Ray Marching (The Shader)

The core logic lives in the Fragment Shader. For every pixel on the screen, we fire a ray. However, instead of a simple vector addition (pos += dir * step), we apply a gravitational deflection force at every step of the march.

We approximate the Schwarzschild metric by modifying the ray's direction vector ($\vec{D}$) based on its distance ($r$) from the singularity:

// Inside the Ray Marching Loop

float stepSize = 0.08 * distToCenter; // Adaptive stepping

vec3 gravityForce = -normalize(rayPos) * (1.5 / (distToCenter * distToCenter));

// Bend the light

rayDir = normalize(rayDir + gravityForce * stepSize * 0.5); 

rayPos += rayDir * stepSize;

Optimization Strategy: Note the stepSize. I implemented adaptive ray marching. We take large steps when the photon is far from the black hole (low compute cost) and micro-steps when close to the event horizon (high precision required). This keeps the loop iteration count low (~120 passes) while maintaining visual fidelity.

2. The Event Horizon & Accretion Disk

We define the Schwarzschild Radius ($R_s$).

• If a ray's distance drops below $R_s$, it is trapped. The pixel returns black (the shadow).

• If the ray intersects the equatorial plane ($y \approx 0$) within specific radii, we render the Accretion Disk.

To simulate the Doppler Beaming effect (where the disk looks brighter on the side moving toward the camera), I calculated the dot product of the disk's rotational velocity and the ray direction.

3. Keplerian Orbital Mechanics (The JavaScript)

The background stars aren't just animating on a linear path. They follow Kepler’s 3rd Law of Planetary Motion ($T^2 \propto r^3$).

I engineered the JavaScript layer to handle the state management of the celestial bodies:

• Blue Giant: Far orbit, period set to exactly 10.0s.

• Red Dwarf: Near orbit.

• Math: I calculated the inner star's period dynamically based on the ratio of the semi-major axes to ensure physical plausibility.

// Keplerian Ratio Preserved float r2 = 7.5; float ratio = pow(r2 / r1, 1.5); // T^2 proportional to r^3 float T2 = T1 * ratio;