Exploring AI-assisted Development, Interaction Design, and Real-time Performance Systems
Creative technology · Rapid prototyping · UX design · AI-assisted development
The Experiment
Could I create an interactive character experience for livestreaming without conventional character rigging, complex streaming automation, or extensive programming?
I explored this question through two browser-based prototypes, each using a different approach to creating responsive, operator-controlled experiences.
The goal wasn't to build another streaming platform. It was to create a lightweight performance layer that could work alongside existing broadcasting software, making it easy for one person to control characters, animations, and visual effects in real time.
Two Approaches
01 — Pixel-Art Fortune Teller
A game-inspired interactive scene featuring animated characters, expressive reactions, crystal-ball readings, and dialogue.
I defined the interaction model, character behaviours, visual direction, and operator controls. Through iterative prototyping with Claude, I explored the limitations of code-generated pixel art, refined character animation, and developed an interaction system that balanced visual personality with practical operation.
02 — AI-Video Character Stage
A second experiment explored replacing programmatic character animation with AI-generated video clips.
I developed a clip-based interaction model built around idle loops, manually triggered actions, automatic state transitions, and costume variations. The system incorporated seamless video switching, visual effects, and motion-tracked accessories.
This approach allowed more visually detailed character performances while retaining a simple, responsive control interface.
Design Decisions
Across both experiments, I focused on making the experience practical to operate.
Human-directed interaction:
Kept the operator in control rather than relying on automated audience triggers.
Separated controls and presentation:
Created a private control console and a clean stage window for broadcasting.
Constraint-led development:
Worked within browser limitations, platform overlays, media consistency requirements, and AI-generation costs.
Progressive refinement:
Evaluated functioning builds, identified usability and visual problems, tested alternatives, and adjusted implementation decisions.
Purposeful use of existing technology:
Used established streaming tools for broadcasting rather than recreating their functionality.
Development Approach
I directed the development process using Claude as an AI coding collaborator, defining requirements, creating visual assets, testing functionality, evaluating alternatives, and refining the user experience.
Claude handled code implementation and technical media processing, allowing me to concentrate on interaction design, visual quality, functionality, and the practical challenges of operating the experience.
Outcome
The experiments produced working interactive browser applications and demonstrated two different approaches to controlling character-based experiences: programmatic animation and AI-generated video.
They also provided a practical opportunity to explore AI-assisted development, multimodal media, interactive systems, and the tradeoffs involved in turning an experimental idea into a usable tool.
