Case study in LLM-API orchestration

CueCode: Natural Language to REST API

Project overview

CueCode is a developer-focused framework that translates natural language into executable REST API payloads using large language models (LLMs). The goal was to make it easier — and safer — for fullstack developers to integrate LLM-powered features into their apps, especially for systems with complex APIs. The timing of CueCode's development highlights Guidelight's lead architect's ability to foresee industry needs and build toward them, as evidenced by Anthropic's convergent development of tools in this problem space.

Project details

  • Project type: Academic capstone
  • Role: Lead developer and architect
  • Project Start Date: Aug 24, 2024
  • Project Completion Date: May 7, 2025
  • Technologies: Python, OpenAPI, SpaCy, PostgreSQL, pgvector, sentence-transformers, OpenAI

Project links

Project website GitHub repository Runtime algorithm code

Problem

While LLMs are increasingly powerful, integrating them into business apps remains risky and technically demanding. Developers face challenges such as:

  • Lack of reusable frameworks for converting text into structured API payloads
  • Risk of hallucinated output or broken request chains
  • Time-consuming prompt engineering and validation logic

CueCode aimed to solve these with a developer-friendly system that supports OpenAPI specs, human-in-the-loop validation, and safe execution of multi-step API tasks.

Solution

Applying prior work (1) on read-only API workloads to write workloads, CueCode bridged the gap between natural language and automated sending of structured, controlled API requests.

CueCode provides a complete framework for text-to-API interaction:

  • Developer Portal to upload OpenAPI specs and configure endpoint logic
  • Python backend that parses natural language and returns suggested REST payloads
  • Client libraries to let apps query CueCode for suggestions via HTTP
  • LLM integration via OpenAI and SpaCy for entity extraction
  • Payload validation and optional human approval workflows

The system was built using 12-factor principles for easy containerization and modular deployment.

Technical highlights

  • Used OpenAPI + prompt engineering to constrain model output and guide generation
  • Applied pgvector + cosine similarity to match ambiguous user intent to API endpoints
  • Developed an execution ordering algorithm to resolve multi-call dependencies
  • Incorporated SpaCy for grammar and named entity recognition
  • Provided two runtime modes: human-in-the-loop and batch auto-execution

Tech Stack Snapshot:

  • Language: Python
  • LLM Host: OpenAI
  • NLP: SpaCy
  • Database: PostgreSQL + pgvector
  • Spec Format: OpenAPI 3.1
  • Architecture: 12-factor, containerized

Results & Impact

CueCode was evaluated using the OpenAPI Petstore API as an API target. Results showed:

  • High match rate for expected payloads
  • Low error rate in chained calls with dependent entities
  • Demonstrated that safe, LLM-driven API generation is achievable using open standards

Importantly, the system lowered the technical barrier for fullstack developers to experiment with and deploy LLM-backed functionality in production environments.

Lessons & Takeaways

  • LLMs + OpenAPI + Validation offer a promising pattern for practical, safe text-to-API workflows.
  • The hardest part isn't generation — it's structural integrity and trust.
  • Even before formal protocols existed, CueCode identified this architecture gap and built toward it.
  • During CueCode's development, Anthropic introduced the Model Context Protocol (MCP) — a formal solution to the same problem space.
  • This convergence validates the architectural instincts behind CueCode, and highlights Guidelight’s ability to anticipate standards and design around them.

References

Bridging the Gap, Zafin engineering.

Build for tomorrow

CueCode's existence prior to MCP reflects a strong architectural forecast and demonstrates that we see beyond the tools of today into the platforms of tomorrow.

Back to Portfolio

Do you need something similar? Get your project started with Guidelight

Contact us