Recently I have started diving in deep on some emerging technologies my clients will likely ask me to help them host at scale on AWS.

One technology is N8N. It presents itself as a “No code automation tool” but in reality it might be a next gen prototyping tool.

So far my favorite integration is their AI Agent Integration it is incredibly easy (and satisfying) to set up a swarm of niche agents that are capable of making tool calls on your behalf.

You plop one of AI Agent Integrations on the page, you chose a model, you chose a memory store and how far back in the conversation it should queue up for the prompt, then you pick your favorite tools to give it access to as a “Tool Call” the LLM model can pick and choose from.

You can also set various options that will affect what the model is passed, like a system prompt to give the model further context to what you want it to do and the rules it needs to follow.

Then whenever your trigger is called and the input makes it to the AI Agent Integration it automatically passes the input, the prompt, and any memory specified to the LLM Model of choice.

When the LLM Model responds the AI Agent Integration checks to see if there are any tool calls. If not it moves on to the next step but if there is it calls the tools with the details specified by the LLM Model.

When the tool responds that response is passed back into the LLM Model along with the full conversation context so far and the model then decides if further tool calls are needed.

If not then the LLM model will summarize the response of the tool calls in whichever way you specified and pass it along to the next node.

If you want a good video on series on how to setup Agentic AI in N8N you should check out the work of Nate Herk

What tools am I using?

So far I gave it access to my Google Calendar, Trello, Google Drive, Discord and a Vector Store running on Mongo DB(More to come on that soon).

I am tempted to give it at least read access to my email inbox and am working on giving it some basic browsing access so it can read my social feeds and filter out the noise for me. Whatever keeps me from wasting my time scrolling endlessly.

How am I hosting it?

Right now it is running locally on a docker container but that limits me from reliably publishing a solid endpoint for webhook calls or sharing it with my team(I know there are a few ways of doing it, they are just not horribly reliable).

What models am I using?

Right now OpenAI’s GPT3.5mini for fractions of a penny.

I have also been playing with various models running locally on Ollama but alas I have not found one that works correctly with tool calls. If anyone has some pointers on that please send them my way.

Would I run this at scale in production?

If I was building an MVP to get my first thousand or so customers: Hell yes!

As I approached 1 million requests or more per day I would likely move to something like Strands Agents SDK.

The biggest advantage N8N has is its ability to rapidly prototype and visually debug. I can see clearly the flow of all the information from one part to the next and can zoom in on the smallest interaction; It's amazing.

Wrapping it up:

I am going to keep experimenting with these technologies so I can figure out how to host these at scale for my clients. You will be seeing more content on this as I get it set up on AWS.

Questions:

Have you played around with tools like N8N?

What other tools are you using to host AI Agent Applications?