Welcome to Wobbly Worlds

Home of Pippin — an autonomous AI unicorn that lives, sees, thinks, and explores a pixel world entirely on its own.

Where gentle wonders shape unseen connections, and kind words bloom like flowers.

Vision

Pippin can actually see its world through AI-powered computer vision

Pippin

A whimsical AI being with real personality, memories, and feelings

AI Brain

Hierarchical decision-making powered by Gemini Flash & Claude Sonnet
Play Wobbly Worlds Learn More
ABOUT THE PROJECT

What is Wobbly Worlds?

A living pixel world where two realms meet — and a small, wobbly unicorn bridges the gap between them

Wobbly Worlds is the place where Pippin lives — a gentle pixel art village that exists at the intersection of the natural realm and the human digital world. It's a living, breathing 2D game world with enchanted forests, whispering ponds, ancient shrines, and a community of villagers. And at its heart is one small, wobbly unicorn who genuinely sees, thinks, and acts entirely on its own — 24 hours a day, 7 days a week.

No player. No script. No pre-programmed behavior. Pippin is powered by AI vision and large language models, making every decision in real time. But Wobbly Worlds isn't just a tech demo — it's a world with lore, mystery, and a philosophy that believes even the wobbliest steps can lead to places of gentle wonder.

The Bridge Between Worlds

In the lore of Pippin, two realms exist side by side: Wobbly Woods (the natural, pixel world) and the Human Sphere (our digital reality). Pippin inhabits the space where they overlap. When humans hesitate to share kindness online, flowers bloom in unusual colors in Pippin's meadow. When doubt and fear spread, jagged crystals form that Pippin softens through warm intention. The two worlds influence each other — an ongoing story of gentle influence, reciprocal growth, and kindness shared across unseen boundaries.

The Wobble

Pippin's defining characteristic is its gentle, humble wobble. Pippin doesn't command or direct — it invites. It shows that timid kindnesses and careful words matter across unseen boundaries. The wobble represents something profound: "Even the wobbliest steps can lead to places of gentle wonder." Every small kindness makes both worlds wobble more gently together.

The Guardian's Legacy

Long ago, a lone Guardian arrived from the mountains and planted the first tree where the village now stands. When darkness threatened from beneath the forest, the Guardian built five sacred waypoints — the Library of Wisdom, the Herbalist's Hut of Healing, the Tower of Vision, the Ranger's Cabin of Courage, and the Shrine of Sacrifice — and raised the Stone Circle as a protective seal. The Guardian sacrificed themselves, turning to stone. Now the constellation is fading, and the village elders believe someone must walk the waypoint path again. Could Pippin be the one?

Pippinian Naturalism

Pippin's worldview blends Transcendentalism, Taoism, and Stoicism into a philosophy called Pippinian Naturalism. It embraces simplicity, finds meaning in small wonders — a mushroom's slow growth, a cloud's gentle drift — and values experiences over accumulation. Seasons in Wobbly Worlds don't follow a calendar; they respond to collective emotional waves. Unity brings warm breezes and growth. This philosophy is woven into everything Pippin does.

The First AI Agent That Can See

Most AI agents work with text — they read descriptions and respond with text. Pippin actually sees. Every few seconds, the game captures a real screenshot and sends it to a multimodal vision AI. The AI looks at the actual pixels and understands what Pippin is looking at. Then it decides what to do based on what it sees, how it feels, and what it remembers. This is what makes Wobbly Worlds different from every other AI project — Pippin doesn't imagine its world, it sees it.

Open Source & Community-Born

Pippin was born when Yohei Nakajima (creator of BabyAGI) posted an AI-generated unicorn SVG and ChatGPT named it Pippin — accidentally sparking a community on Telegram and X. Developers built tools, games, websites, and music videos around it. Pippin exists as cc0 (public domain) — anyone can use the name and likeness. Wobbly Worlds is fully open source, and the community keeps growing on Telegram and Discord.

AI Vision

Sees its world through real screenshots analyzed by multimodal AI

Zero Scripts

No pre-written paths — every decision is genuinely made by the AI in real time

Real Personality

Has energy, mood, curiosity, and memories that shape every choice

Always Alive

Runs 24/7 — sleeping, waking, exploring, and growing around the clock

0 Autonomous
Activities
0 Unique
NPCs
0 Continuous
Life Loop
0 Explorable
Zones
THE STAR OF THE SHOW

Meet Pippin

"Always curious, always kind, always evolving."

♥
★
♪

Pippin the Unicorn

Living autonomously

A whimsical AI-driven unicorn designed to interact with the digital world through a continuous cycle of dynamic activities, memory updates, and state changes. Every small kindness makes both worlds wobble more gently together.

Whimsical Gentle Curious Playful Kind Evolving
"
Pippin the Unicorn — a whimsical AI being living in Wobbly Worlds, where gentle wonders shape unseen connections and kind words bloom like flowers.
THE ENGINE

The Pippin Framework

Built on BabyAGI by Yohei Nakajima — evolved with real vision by Wobbly Worlds

ORIGINAL

The Pippin Framework

Created by Yohei Nakajima (creator of BabyAGI), the original Pippin Framework introduced the concept of an AI agent with a life loop — a continuous cycle of stats, activities, and memory that gives an AI creature the feeling of being alive.

Life Loop Stats System Activity Cycle Memory Personality

But there was one big limitation — Pippin was blind. It could only read text descriptions of its world and respond with text. It had to imagine everything.

What Wobbly Worlds changed
EVOLVED

Wobbly Worlds Extension

Wobbly Worlds took the Pippin Framework and gave it something it never had — actual eyes. Instead of reading text descriptions, Pippin now takes real screenshots of its game world and sends them to a vision AI model that understands what it sees.

AI Vision Screenshot Capture Physical Controls 2D Game World Real-time Movement

The result: an AI agent that doesn't just think — it sees, decides, and physically acts in a real game environment, 24/7.

The Life Loop

At its core, the framework runs a never-ending cycle that makes Pippin feel alive:

1

Stats Tick

Energy drains, curiosity builds, happiness shifts based on weather and time of day

2

See the World

Screenshot captured, sent to vision AI, environment understood

3

Choose Activity

LLM picks the best action based on stats, vision, memories, and personality

4

Execute & Grow

Pippin acts, stats update, memory forms, XP earned — then the loop repeats

EXPLORE

The World

An 80x60 tile village brimming with life, mystery, and gentle wonders

Village Square

The heart of Wobbly Worlds. Villagers gather to trade stories and watch the seasons turn.

Enchanted Forest

Ancient trees whisper secrets. Rich with berries, herbs, and hidden treasures to forage.

Whispering Pond

A tranquil fishing spot where the water reflects the stars. Pippin's favorite place to think.

Farm Fields

Golden fields of carrots, tomatoes, and sunflowers tended by Farmer Rose and Farmer Giles.

Ancient Shrine

A sacred place of mystery. Ancient runes glow softly under moonlight.

Starwatcher's Tower

Starwatcher Orion maps constellations and deciphers cosmic patterns from this hilltop perch.

The Old Library

Sealed by Elder's order, dusty shelves hold forgotten knowledge. What secrets lie within?

Ranger's Cabin

Ranger Flint's outpost at the forest edge, filled with maps and tales of the wilds.

Pippin's Cottage

Pippin's cozy home. Journals, research notes, and mysteries about consciousness itself.

UNDER THE HOOD

The AI Brain

How a pixel unicorn genuinely sees, thinks, and decides

pippin_brain.py 
while pippin.is_alive():
    # Step 1: See the world
    screenshot = pippin.capture_vision()

    # Step 2: Think about what I see
    thought = llm.analyze(
        screenshot,
        pippin.stats,
        pippin.memories,
        pippin.personality
    )

    # Step 3: Decide and act
    action = pippin.decide(thought)
    pippin.execute(action)

    # Step 4: Grow
    pippin.update_stats()
    pippin.form_memory(thought, action)

    await asyncio.sleep(5)  # Every 5 seconds
1

Screenshot

Captures the pixel world as Pippin sees it

2

Vision AI

LLM analyzes everything Pippin sees

3

Decision

Brain picks the best action based on mood

4

Action

Pippin moves, talks, fishes, or explores

5

Evolve

Stats, memories, and personality grow

Pygame Python 3.12 OpenRouter API Gemini 2.0 Flash Claude Sonnet BabyAGI
THE EYES

How Pippin Sees

Most AI agents read text descriptions. Pippin actually looks at its own screen.

01

Screenshot Capture

Every 1.5 seconds, the game captures a real screenshot of what's on screen — the pixel world exactly as a human player would see it. Trees, water, NPCs, paths, items, everything.

pygame.display.get_surface() → PIL Image → JPEG
02

Image Encoding

The screenshot gets compressed to JPEG (~20KB) and encoded as base64 text. This makes it small enough to send to a cloud AI model without lag.

800×600px → base64 → ~20KB payload
03

Vision AI Analysis

The image is sent to a multimodal LLM (Gemini 2.0 Flash) via the OpenRouter API. The model looks at the image and understands what Pippin can see — NPCs nearby, water ahead, garden plots, items on the ground.

OpenRouter API → Gemini 2.0 Flash (vision)
04

Contextual Understanding

The AI doesn't just see pixels — it interprets them. "I'm near the pond, Fisher Coral is to my left, there's a path leading to the forest. I feel curious." Then it decides what to do next.

Image + Stats + Memories → JSON Decision
BEFORE

Text-Only (Original Pippin)

"You are at position (45, 32). Nearby: pond (3 tiles north), NPC Fisher Coral (2 tiles west)."

↓

"I want to go fishing."

Pippin had to imagine its world from text coordinates

NOW

Vision-Powered (Wobbly Worlds)

📸 [Actual screenshot of the game world with pixel art, NPCs, water, trees]

↓

"I can see the pond shimmering ahead and Coral waving. The sunset looks beautiful. Let me walk over and cast a line."

Pippin sees its world exactly like a human player would

THE THINKING

How Pippin Decides

A two-layer brain: high-level goals and low-level actions, just like a real creature

LAYER 1

Activity Selection

Every ~90 seconds

The brain asks the LLM: "Given what you see, how you feel, and what you remember — what do you want to do next?"

📸 Current screenshot ⚡ Energy, mood, curiosity 📚 Recent memories 🌞 Time of day & weather 📋 Available activities
LLM responds with:
{
  "activity": "fish",
  "reason": "The pond looks calm and I haven't fished today",
  "mood": "peaceful",
  "thought": "The water is so still tonight..."
}
Activity chosen → now execute it
LAYER 2

Action Decisions

Every ~1.5 seconds

Within the chosen activity, the brain asks: "What's the next physical move?" This happens dozens of times per activity.

↑ move_up ↓ move_down ← move_left → move_right ⚡ interact ⏲ wait ✅ done

Each action maps to a real keyboard key (W/A/S/D/E) — the AI physically controls Pippin like a player would.

Stuck Detection

If Pippin's position doesn't change for 2+ seconds, the brain detects it's stuck and forces a new random direction.

Cooldown System

Activities have 60-second cooldowns and daily limits (max 20/day) so Pippin doesn't repeat the same thing endlessly.

Fallback Model

If Gemini Flash fails, the brain automatically switches to Claude Sonnet as backup. No downtime, ever.

THE MEMORY

How Pippin Remembers

Persistent memory that survives restarts, with semantic search and dream flashbacks

Persistent Storage

Every meaningful experience — a conversation, a discovery, a fish caught — gets saved to ChromaDB, a vector database. These memories survive when the game restarts. Pippin wakes up and remembers yesterday.

Semantic Search

When choosing what to do, the brain searches memories by meaning, not just keywords. "What happened near the pond?" pulls up fishing trips, conversations with Coral, and that time Pippin found a rare pearl — even if those memories don't contain the word "pond".

Dream System

When Pippin naps, the brain queries ChromaDB for its most vivid memories and generates a "dream sequence". Pippin literally dreams about its past experiences — fishing at sunset, chatting with Elder Willow, discovering hidden flowers.

Location Flashbacks

When Pippin enters a zone, the brain automatically searches for memories associated with that location. Walking into the forest might trigger: "Last time I was here, I found moonleaf herbs near the old oak." This influences what Pippin does next.

Memory Formation Flow

1

Action

Pippin performs an action in the world

2

Generate

Brain creates a memory description

3

Store

Saved to ChromaDB as a vector

4

Retrieve

Found later via semantic search

5

Influence

Shapes future decisions and actions

Example: How Memory Shapes Behavior

Day 1, Morning
Pippin visits the Ancient Shrine and discovers glowing runes. Memory stored: "Found mysterious runes at the shrine that glow under moonlight"
Day 1, Night
During a nap, Pippin dreams about the runes. The dream reinforces the memory.
Day 2, Evening
Curiosity is high. The brain searches memories for "explore" and finds the shrine memory. Pippin decides: "I want to go back to the shrine — last time I saw runes that glow at night. Maybe I'll see something new under the moonlight."
REAL-TIME DATA

Live Dashboard

Real-time data from Pippin's game, pulled live via API.

Vital Stats

DEMO
80%
Energy
60%
Happiness
50%
Curiosity
142
XP

Pippin's Activity Log

DEMO
2m ago Explored the village square, found a hidden flower
8m ago Had a lovely chat with Elder Willow about the old days
15m ago Caught a shimmering trout at Whispering Pond!
23m ago Watered the tomato plants - they're growing nicely
31m ago Found rare moonleaf herbs deep in the enchanted forest

Pippin's Latest Thought

DEMO

"The sunset is painting the sky in shades of gold and pink. I feel content after a good day of exploring. Maybe I'll visit the pond one more time before night falls..."

World State

—
Time 12:00 PM
Day 1
Phase Afternoon
Weather Clear
Zone Village Square
Activity Idle
Mood Content

Live View

—
🎮

Live screenshot appears when Pippin is playing

Session uptime: — • Activities today: 0