From API Keys to Local Power: Run Your CrewAI Agents Offline

 




In the previous post, I shared my update about building collaborative AI agents with the CrewAI framework. We explored the power of orchestrating AI agents to tackle complex tasks. However, that setup relied heavily on powerful Large Language Models (LLMs) accessible through expensive API tokens.

Since then, I've discovered a transformative approach: running CrewAI agents entirely locally. This shift leverages tools like Ollama and open-source models such as DeepSeek. The method was brilliantly demonstrated by Tyler AI in his YouTube video "How to Build CrewAI Agents with Deepseek R1 100% Locally".

The Economic and Practical Advantages of Local LLMs

As Tyler AI emphasizes, the financial contrast between API-driven models like OpenAI's gpt-3.5-turbo-016 (around $60 per million output tokens) and open-source alternatives like DeepSeek R1 (roughly $2.19 per million output tokens) is substantial. Impressively, DeepSeek R1 achieves performance comparable to or even surpassing certain OpenAI models in various benchmarks. Additionally, DeepSeek offers fully open-source models, including distilled versions that require fewer computational resources. This makes deploying sophisticated AI workflows on your hardware an economical and feasible solution.

The cornerstone of this local setup is Ollama, a user-friendly tool that simplifies downloading, running, and managing LLMs locally. With Ollama, CrewAI can seamlessly connect to these locally hosted models.

Streamlined Steps for Local CrewAI with DeepSeek and Ollama

Based on the process detailed in Tyler AI's video, here’s how to update your CrewAI code for local LLM usage:

1. Effortless Ollama Installation:

  • Download and install Ollama from Ollama’s official website.
  • Follow the installation process and launch Ollama. This will initiate the necessary local server to host your LLMs.

2. Downloading Your Local LLM with Ollama:

  • With Ollama active, download the LLM you intend to use. For example, if you’re running this on a 2015 Mac Mini with no GPU, use the base model DeepSeek-r1:1.5b.
  • If you have a more powerful machine, consider using the 5b or 15b parameter models.
  • In your terminal, enter: 
    ollama pull deepseek-r1:1.5b

3. (Optional but Recommended) Verifying Your Local LLM with Ollama:

  • Before integrating with CrewAI, ensure the model functions correctly by running: 
    ollama run deepseek-r1:1.5b
  • This allows you to interact with the model to confirm its operational status.

4. Modifying Your CrewAI Code for Local LLM Integration:

  • Open your crew.py file and include the following configuration:
from crewai import Agent, Crew, Process, Task, LLM

deepseek_ollama = LLM(
    model="ollama/deepseek-r1:1.5b",
    base_url="http://localhost:11434",
)
  • The prefix ollama/ before the model name ensures the LLM class connects to the Ollama server.

5. Running Your Locally Powered CrewAI Script:

  • Start your Ollama server and run your CrewAI script normally. Ensure Ollama is active before executing the script to avoid connection errors.
  • Be aware that the initial run may take longer as the model loads into your machine’s memory.

Benefits of Local AI Agents

  • Cost Reduction: Avoid the recurring expenses associated with API token consumption.
  • Enhanced Privacy: Your data and AI processing remain confined to your machine, providing greater control over privacy.
  • Complete Control: You have full authority over the LLMs being used, allowing for customization and optimization according to your needs.

By embracing this local approach, you gain complete ownership of your intelligent agent applications while dramatically reducing costs and enhancing privacy.

Comments

Post a Comment

Popular posts from this blog

The Talent Gap Myth: How Traditional Engineers Evolve in the GenAI Era

Image
The Talent Gap Myth Generative AI has created a perception that an entirely new set of roles is emerging — “prompt engineers,” “AI whisperers,” “autonomous-agent designers.” That narrative, while exciting, is incomplete. In reality, the foundations of GenAI systems are still deeply rooted in software engineering discipline . The roles are evolving, not disappearing. What changes is the interface: engineers now work with language and context instead of code and schema . The future won’t be built by replacing traditional engineers — it will be built by those who learn to translate their existing skills into this new paradigm of adaptive, context-aware systems. This article kicks off a series exploring how traditional software roles are evolving in the GenAI era. Over the next few posts, we’ll deep-dive into each transition — backend to context architecture, frontend to AI interaction design, QA to hallucination testing, and more. Backend Engineer → Context Architect / LLM Integrator Th...
Read more

Traditional AI vs. Generative AI: Same Foundations, Different Language

Image
There’s a lot of buzz around Generative AI, with many touting it as a new computing paradigm — but that misses the point. GenAI and Agentic AI are still software systems , built on the same engineering discipline that’s powered the IT industry for decades. Both follow principles of modularity, clear responsibility, orchestration, and observability. What’s changed isn’t the foundation — it’s the interface . Intelligence is now expressed through natural language instead of code, and coordination happens through context instead of strict APIs. The Core Hypothesis Both Traditional and Generative AI share the same architectural backbone: modular components with clear boundaries and predictable interactions. What’s evolved is the language of communication — from code contracts to context contracts. Deterministic logic has given way to probabilistic reasoning, but the need for disciplined engineering remains unchanged. Parallels Between Traditional Systems and Agentic AI a. Single...
Read more