Teaching AI to Contribute to Open Source: Lessons from MapLibre Agent Skills
2026-09-02 , Ran2

Open-source communities can't afford to ignore the growing use of AI agents to write code. This talk shares lessons from building MapLibre Agent Skills: an open, eval-tested knowledge base that corrects AI hallucination in MapLibre implementations, with lessons for other projects looking to do the same.


You've heard about the AI code contribution problem: maintainers flooded with plausible-but-wrong pull requests and issues generated by AI assistants that don't understand the codebase. But there's a second problem that gets less attention: AI doesn't just generate bad contributions — it generates bad advice. Developers building with open-source libraries are getting confidently wrong answers, and the resulting confusion circles back as bug reports, forum questions, and more maintenance load.

Agent skills — curated knowledge files that any AI coding assistant can load into context — offer open-source projects a way to address both problems at once. They improve the code AI writes for your users, which reduces the confused issues that reach maintainers. They can also raise the floor on AI-assisted contributions by giving models accurate, version-specific knowledge about your project's APIs. Proprietary platforms ship agent skills as a commercial product alongside their paid services. Open-source projects have to build them differently: in the open, with community authorship, and so far sustained with a non-existent budget. Yet the value is clear: they give LLMs the knowledge to get it right the first time: fewer wrong answers, fewer confused issues, fewer wasted review cycles.

MapLibre Agent Skills is one such effort. Hosted under the MapLibre GitHub organization, it's a growing collection of skills in the open agent skills format supported by every major AI coding assistant. Each skill targets a topic where AI regularly fails MapLibre developers — confusing MapLibre APIs with Mapbox's, or hallucinating method signatures that changed between major versions. We identify these failure points through systematic mining of GitHub issues, Stack Overflow questions, and community Slack conversations — the places developers land after AI gives them a wrong answer.

But writing skills is only half the problem. How do you know they work, and how do you know they create solutions that can keep working? We built an evaluation pipeline that tests every skill for grounding, completeness, code correctness, and reliability — but the harder questions are the ones we can't answer with automated tests. Will developers keep asking questions on Stack Overflow and community forums, or will they give up where they can't blindly trust coding assistants? Will the companies who use and integrate these tools fund the maintenance work and ecosystem development that keeps documentation current?

This talk shares what we've learned building MapLibre Agent Skills and names the open questions we're still working through. If you build with MapLibre, you can install these skills today. If you have expertise in any corner of the ecosystem — framework integration, tile pipelines, cartography, migration — we're looking for skill authors. And if you're wrestling with these same questions about AI in the open source world, bring them. This conversation is bigger than any one project, and transparency, collaboration, and cooperation are how we'll figure it out.


Level of technical complexity: 2 - intermediate Give indication of resources (video, web pages, papers, etc.) to read in advance, that will help get up to speed on advanced topics.:

MapLibre Agent Skills Repo — https://github.com/maplibre/maplibre-agent-skills
OpenAI: "Testing Agent Skills Systematically with Evals" — https://developers.openai.com/blog/eval-skills/
Minko Gechev: "Unit Tests for AI Agent Skills" — https://blog.mgechev.com/2026/02/26/skill-eval/
Anthropic: "Demystifying Evals for AI Agents" — https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
RAGAS metrics — https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/
Promptfoo evaluate coding agents — https://www.promptfoo.dev/docs/guides/evaluate-coding-agents/

Indicate what is (are) the open source project(s) essential in your talk:

MapLibre

I make my conference contribution available under the CC BY 4.0 license. The conference contribution comprises the abstract, the text contribution for the conference proceedings, the presentation materials as well as the video recording and live transmission of the presentation: