AI Video Ads vs Human UGC: What $100K in Testing Revealed

Everyone has an opinion on whether AI can replace human creators for video advertising. Most of those opinions are based on vibes, not data. We decided to settle the debate with cold, hard numbers.

Over six months, we ran $100K in ad spend across three distinct creative approaches: pure AI-generated ads, AI-assisted human ads, and traditional human UGC. The results destroyed several assumptions we held going in.

The Test Setup

We tested across three DTC brands in different verticals: skincare, fitness supplements, and home organization. Each brand had established baseline performance metrics, giving us clean comparisons.

Three creative approaches tested:

• AI-Only: Scripts written by AI, performed by AI avatars (Creatify Aurora and Arcads), edited with AI tools. Zero human involvement in production.
• AI-Assisted: Human strategists wrote briefs and selected hooks. AI generated scripts and variants. Human creators performed key emotional beats, with AI handling B-roll, captions, and optimization.
• Human UGC: Traditional UGC workflow. Real creators, real scripts (human-written), real production. The control group.

Each approach got equal ad spend. We tested across Meta, TikTok, and YouTube Shorts. All ads ran to cold audiences with identical targeting.

The Raw Numbers

Return on Ad Spend (ROAS)

The headline number tells one story: human UGC slightly edges out AI-assisted, and both crush AI-only. But the real insights are in the details.

Cost Per Acquisition (CPA)

Here is where it gets interesting. Human UGC and AI-assisted had nearly identical CPAs, but AI-assisted production costs were 75% lower. When you factor in total cost (production plus media spend), AI-assisted was the clear winner on unit economics.

Engagement Metrics

AI-only ads hooked fewer people and held even fewer. The uncanny valley is still real, even with the latest avatars. People scroll past AI faces faster than human ones. But AI-assisted content actually had a higher hook rate than pure human UGC, likely because AI-generated hooks were tested at higher volume before production.

The Five Key Takeaways

1. AI-Only Is Not Ready for Premium Direct Response

At 2.1x ROAS, AI-only ads are profitable but barely. The 0.8x worst-case scenario means you can easily lose money. The variance is too high for brands that need consistent, predictable returns. AI-only works for brand awareness at scale, but not for performance marketing where every dollar needs to pull its weight.

2. AI-Assisted Is the Sweet Spot

The 4.7x ROAS with 75% lower production costs makes AI-assisted the clear winner when you look at total return on investment. Human strategic thinking combined with AI execution speed creates something neither can achieve alone.

3. Testing Volume Is the Real Advantage

Here is what the ROAS numbers do not capture: with AI-assisted workflows, we produced 12x more ad variants in the same timeframe. The best AI-assisted ad (7.2x ROAS) outperformed the best human ad (6.1x). Why? Because we had more shots on goal. More variants means more chances to find a winner.

4. Hook Quality Matters More Than Production Quality

The AI-assisted approach had the highest hook rate despite not having the highest production value. This confirms what direct response advertisers have known for years: the first 3 seconds matter more than everything else combined. AI excels at generating and testing hooks at scale.

5. Platform Matters

Breaking down by platform revealed something unexpected:

• TikTok: AI-only performed closest to human (1.8x vs 3.9x). TikTok audiences are more forgiving of lower production quality.
• Meta (Facebook/Instagram): AI-assisted dominated (5.2x vs 4.4x human). The algorithm rewards creative diversity, and AI-assisted produced more variants.
• YouTube Shorts: Human UGC won decisively (5.8x vs 3.9x AI-assisted). YouTube audiences seem to value authenticity more.

What This Means for Your Strategy

If you are spending under $5K per month on ads, AI-only tools give you a way to compete. The production cost savings are real, and 2.1x ROAS is still profitable.

If you are spending $5K-$50K, AI-assisted is the obvious play. Invest in a strong strategist, use AI for production and testing, and you will outperform most competitors who are still using traditional workflows.

If you are spending $50K and above, run a hybrid approach. Use AI-assisted for your testing and iteration loop, and invest in premium human UGC for your proven winners. Once you identify a winning angle through AI testing, have a real creator produce a polished version.

The Bottom Line

AI did not beat humans. But AI made humans better. The future of ad creative is not AI or human. It is AI multiplied by human. The brands that figure out this multiplication are the ones that will dominate the next era of digital advertising.

The $100K we spent on this test was the best investment we made all year. Not because of the ads it produced, but because of the clarity it provided on where to allocate creative resources going forward.