--- published: true layout: post title: Challenges with Automating API Profiling tags: - Profiling - Reviews - Discovery image: >- https://kinlane-productions2.s3.amazonaws.com/algorotoscope-master/america-immigration_dumping-ground-banana-subway-stairs.jpeg --- There are so many differences between the APIs out there. Even when they are HTTP, Web, or RESTful APIs. I know that people feel like APIs can just be automated and AI will take care of connecting and publishing APIs, but this is a reality you buy into when you haven’t spent much time in the trenches producing or consuming APIs. I am pragmatically using AI and automating wherever possible when it comes to the discovery, harvesting, processing, and refinement of APIs for inclusion in the APIs.io search engine I am slowly building. I guarantee that nobody has spent as much time as I have discovered and onboarding with APIs, and I am very, very, very aware of the reality that we won’t be able to automate the majority of this work anytime soon. ## Technical Details When you get hands-on with an API crafting OpenAPIs and making executable using Bruno or Postman Collections you find a new level of intimacy with consistent technical details like paths, parameters, and status codes matter. Just look at the difference between the Threads API, Bluesky API, and Mastodon API. Three separate APIs seemingly doing the same thing, but in three wildly different ways—two of which claim to be a standard designed for interoperability and federation. There are some whacky takes on REST present in those APIs, as well as deviations towards RPC. In my experience very few people conduct any due diligence before designing an API, or even an API standard, before they get to work. If I was building an SDK that worked across all three of these APIs, one may start out optimistic about the time and energy it would take, but then after months you’d realize that it just isn’t cost effective and not worth your time. ## Business Details The way that you discover, learn about, onboard, and keep up with the changes across any API platform is just a Wild West. I know most technologists don’t see these things and take them for granted, but they matter and result in real time wasted and money spent. Figuring out where to sign-up, get your keys, and understand common errors is up for grabs. Is this free? Will I have to pay for it? What are my rate limits? Oh, and what are the legal implications of my usage and the data I publish to this API? None of this is standardized and all of it uses a varying vocabulary to describe processes and expectations in a way that often leaves struggling to understand what is going on. I can only imagine what AI will hallucinate when trying to navigate these waters. I would say that the technical details are closer to being standardized than the business details, because we are closer to the heat, but they are both way off from any sort of common and shared reality—and it is getting worse, not better. ## It Is The Little Things The number of ways in which people draw the lines for bounded context between their APIs, or not at all makes it near impossible to harvest API docs. I can take specific pages and copy / paste the HTML into ChatGPT and get a pretty coherent OpenAPI 3.1 these days. However, that identification of what is an API, and where the lines are, will remain a human thing until services and tooling standardize it. Everywhere you see standardization by tooling, like with Amazon Web Services, you begin to see the limitations, and there are numerous special snowflake AWS API, even with the majority being cookie cutter. And even when you have keys, JWT, and OAuth standardization via the OpenAPI schemes property, where you obtain tokens, certificates, and how you renew is all over the place. Little tweaks in error messages, rate limits, schema, and other things seem minor in the context of a single API, but then across hundreds or thousands of APIs, it becomes very costly and time consuming. ## Jiro Dreams of Sushi I remember watching the [documentary Jiro Dreams of Sushi](https://www.imdb.com/title/tt1772925/), where there was an apprentice who spent years making a little egg dish over and over until he graduated to being able to make actual Sushi, or another incremental part or piece of the overall experience. If I had my way, before you could ever develop, distribute, or evangelize an API, you’d have to spend years discovering, onboarding, and integrating with other people’s APIs. There are endless lessons available here. I have been doing this for 15 years straight and I am still learning new things each day. Being a consumer of APIs is the most valuable experience in this game, and in my opinion where the most power resides. However, being a producer of APIs is where the money and glory is at, and where most of the API storytelling occurs. Producing APIs is where the power gets wielded the most, but in my experience is not where the true power and learning resides—it is in putting them to work. I challenge anyone who believes that artificial intelligence is going to solve the API integration and interoperability problem to come spend a full week with me profiling public APIs. Not just profiling the business and technical details of APIs, but actually onboarding and certifying all of the operations with a Bruno collection. After a week you will still be wet behind the ears and still have so much to learn, but I guarantee you will begin to see the disorder that exists and how this is really a people issue, and not just a technical or business one. I will keep working to automate and use AI in my API profiling and review work. I am not anti-AI when it comes to this stuff, but I am honest and pragmatic about what it can be used for. So far AI will write summaries and descriptions for me about 60% of the time, and generate an OpenAPI 3.1 spec for me when I carefully copy and paste API docs and carefully engineer it at the prompt (look ma I am a prompt engineer), but not much else. I’ll keep pushing for automation at this layer, but so far, very little of that automation is AI, and I still get a lot of return on a nice stack of if then statements and a CRON job.