The latest episode of Using AI at Work features Peter Swimm, founder of Toyleville and former Microsoft Copilot designer, sharing insights from his radical experiment: running his entire business using only voice AI for one week. Host Chris Daigle explores how this experience led to revolutionary approaches for business automation and the potential end of traditional software applications.
From Microsoft to Voice-First Operations
With 25 years in tech spanning data centers, customer service, and Fortune 500 companies, Swimm brings unique expertise in conversational AI. His work at Microsoft on Copilot products gave him insider perspective on the gap between AI capabilities and real-world business implementation. This experience convinced him that small and medium businesses needed better access to AI expertise without requiring massive technical teams.
The revelation came during his week-long voice-only experiment, where he discovered that traditional typing and clicking interfaces were actually barriers to effective AI collaboration. By switching to conversational workflows, he found a more natural rhythm that felt like “co-working with a friend” rather than wrestling with complex prompts.
The Voice-Only Business Experiment
Swimm’s experiment started frustratingly but evolved into something transformative. Initially, the AI generated over 3,000 scripts in three days – an overwhelming flood of automated solutions. The breakthrough came when he developed a “sifting system” that analyzed these outputs, identifying patterns and consolidating them into reusable tools.
By the end of the week, those 3,000 scripts had been refined into 200 unique functions, which further distilled into just 12 essential tools with five core functions each. This process created a personalized AI assistant that could handle morning routines: checking email, reviewing yesterday’s incomplete tasks, analyzing relevance, and preparing daily priorities – all triggered by simply saying “good morning.”
The system learned Swimm’s writing style and tone from his existing Notion content, enabling it to draft communications that required minimal editing. More importantly, it eliminated the constant context-switching that plagues modern knowledge work, allowing deep focus periods protected from digital interruptions.
Rethinking Software Architecture
The experiment revealed fundamental flaws in current business software design. Traditional applications try to serve 99% of users with generalized features, resulting in bloated interfaces filled with unused buttons and functions. Swimm argues that most apps are essentially “the same thing underneath” – web browsers with different user interfaces.
His vision involves AI-powered transformation layers that allow teams to work with their preferred tools while maintaining seamless integration. When his project manager updates a project name in Notion, it automatically propagates through Slack, GitHub, and other connected systems. Team members can work in whatever environment suits them best, with the AI handling translation and synchronization.
This approach eliminates the need for universal platform adoption. Instead of forcing everyone to use the same tools, businesses can leverage individual preferences while maintaining operational coherence through intelligent automation.
The Economics of AI-First Business
Swimm’s lean team structure demonstrates the economic potential of AI-augmented operations. With just a senior developer, a project management officer, and himself, plus contractors for specific tasks, Toyleville can handle multiple client engagements monthly. The AI amplifies each team member’s eight to twelve hours per week into significantly greater output.
This model shifts business dynamics from hunting “big fish” clients to building sustainable relationships more like realtors or barbers – serving clients multiple times with reusable content and processes. The result is a more natural business rhythm that accommodates work-life integration rather than the traditional separation.
Local model deployment presents another economic advantage. While cloud-based AI APIs can cost $150 daily for intensive use, running models locally on modern hardware provides acceptable performance for most business applications. Swimm successfully benchmarks local models on an M4 MacBook with 32GB RAM, proving that sophisticated AI capabilities don’t require enterprise-level infrastructure investments.
Personalization vs. Platform Lock-in
The conversation touches on a critical tension in AI development: major tech companies need comprehensive user data to justify their massive model investments, while users increasingly value privacy and control. Swimm advocates for personal AI systems that function like “solar panels for your house” – providing ongoing value without external dependencies once installed.
His approach involves creating user profiles that capture individual work patterns and preferences without sharing data with external platforms. This personal AI assistant can adapt to different environments – whether working from home, a hotel, or a client office – while maintaining familiar workflows and interfaces.
The vision extends to truly portable computing where a user’s entire digital environment travels with them via a simple key or device, eliminating the friction of adapting to different hardware and software configurations.
Implementation Strategy for Non-Technical Users
For business professionals without technical backgrounds, Swimm recommends starting with a simple mindset shift: instead of asking AI direct questions, explain what you’re trying to accomplish and ask what tools it would recommend. This approach often reveals simpler solutions than the complex workflows people initially envision.
The key insight is moving from brand loyalty to outcome focus. Rather than forcing integration between specific platforms, consider what you actually need to accomplish and let AI suggest the most efficient path forward. This often reduces software dependencies from dozens of applications to a handful of essential tools.
Swimm’s methodology involves treating AI like a research partner throughout the development process. Every business task becomes an opportunity to learn and optimize, creating continuous improvement cycles that compound over time.
The Future of Work and Applications
Looking ahead, Swimm predicts the eventual obsolescence of traditional applications in favor of AI-generated, task-specific tools. Instead of using Excel with hundreds of unused features, users will request exactly the functionality they need, when they need it.
This shift challenges fundamental assumptions about software licensing, subscription models, and user interfaces. When anyone can generate custom solutions for specific problems, the value moves from access to tools toward expertise in problem-solving and system design.
The implications extend beyond software to organizational structure. As AI handles routine coordination and data processing, businesses will prize creative problem-solving, intuitive decision-making, and human judgment over technical skills or process management.
Ready to explore conversational AI workflows for your business? Listen to the full episode for detailed insights on implementing voice-first operations and building sustainable AI-augmented business models.
