Building a Paper-Trading Bot, Part 1: What I Learned From Getting It Wrong Three Times

This is Part 1 of a 3-part series on building a paper-trading bot from scratch. This post is the story — what I built and what went wrong along the way. Part 2 is a step-by-step tutorial with the full working code if you want to build it yourself. Part 3 covers adding a live performance dashboard served over Apache.

I set out to build a trading bot. I ended up learning more about testing discipline, benchmark comparisons, and the subtle ways “correct-looking” code can still be wrong than I did about trading itself. This is the story of that build — the architecture, the backtests that punctured my assumptions, and two live bugs that only showed up once real automation was running unattended.

If you’re thinking about building something similar, I’d rather show you where it actually went sideways than pretend it was smooth from commit one.

The starting point

The goal was modest on purpose: get a Python bot paper-trading on Alpaca, with a strategy simple enough to reason about — a moving-average crossover. Before writing a line of strategy logic, I laid out the pipeline as separate, swappable pieces:

data_feed.py    → pulls historical/latest bars from Alpaca
strategy.py     → turns price data into BUY/SELL/HOLD signals
risk.py         → position sizing and stop-loss logic
executor.py     → submits orders to the paper account
trade_logger.py → records every trade and daily equity snapshot
backtest.py     → runs the strategy against historical data, no live calls
main.py         → the live polling loop

Keeping these separate turned out to matter a lot more than I expected. Every bug I hit later was isolated to one module, and I could reason about (and fix) each one without touching the others.

The first version of the strategy was a plain SMA crossover: buy when a 20-day moving average crosses above a 50-day average, sell on the reverse. Dead simple, and a reasonable scaffold to prove the plumbing worked end to end.

Backtest #1: humbled by buy-and-hold

The first backtest ran fine — positive return, plausible-looking trades. It felt like a win right up until I added one more line of output: what would buy-and-hold have returned over the same period?

That single comparison changed everything. On AAPL over a 730-day window, the strategy returned about 25%. Buy-and-hold returned nearly 47%. The strategy wasn’t just “okay” — it was leaving almost half its potential return on the table, entirely because it sat in cash during a strong bull run, waiting for confirmation signals that a buy-and-hold investor never needed.

Lesson: a backtest without a benchmark isn’t really a backtest — it’s just a number. Positive returns feel good in isolation, but “good” is only meaningful relative to the simplest alternative: doing nothing.

Adding a trend filter — and testing it honestly

The obvious next step was gating the crossover on trend strength, using ADX (Average Directional Index) — only trust a crossover if ADX confirms the market is actually trending, not just noisy. That cut down on whipsaw trades in choppy periods.

But here’s the thing about a filter like this: it has a real cost as well as a real benefit, and you only see the cost by testing it somewhere it’s supposed to hurt. So instead of only testing on one stock, I ran the same strategy across four very different regimes:

AAPL and SPY — both in sustained bull runs over the test window
ACN — down roughly 45% from its highs, a genuine bear market
COIN — volatile and declining, a stress test for both trend and risk logic

The pattern that emerged was completely coherent, and completely different from what a single-symbol test would have shown: the strategy underperformed buy-and-hold by 20-30 percentage points on AAPL and SPY (the cost of waiting for confirmation in a market that just kept going up), but beat buy-and-hold by over 25 percentage points on ACN — it correctly stepped aside during the worst of a real decline.

That’s not a strategy that “works” or “doesn’t work.” It’s a strategy with a specific, identifiable regime it’s good for. Knowing that is worth infinitely more than a single aggregate number from one stock.

The stop-loss that made things worse

Position risk was still an open problem — a naive flat 2% stop-loss sounded reasonable on paper. It was not reasonable in practice.

Backtesting it on COIN revealed why: nine stop-loss exits in under a year, several just days apart. A 2% move is ordinary daily noise for a stock as volatile as COIN — the stop wasn’t protecting against real reversals, it was getting shredded by the stock’s normal breathing room, forcing repeated re-entries at worse prices each time.

The fix was to scale the stop distance to each symbol’s own volatility using ATR (Average True Range) instead of a flat percentage — entry_price - (2.5 × ATR at entry). A volatile stock naturally gets a wider stop; a calm one gets a tighter one. Re-running the same COIN backtest after the change: stop-loss exits dropped from nine to two, and the damage from each was cut by more than half.

Lesson: risk parameters that “feel” reasonable in isolation can be actively harmful without accounting for the asset’s own behavior. Test the thing you added, not just the thing you started with.

Going live surfaced a bug backtesting couldn’t catch

Backtesting works because you’re replaying finalized, settled data. Live trading exposed a class of bug that simply doesn’t exist in that world: timing.

The original signal logic only fired a BUY on the exact bar a crossover happened, gated by whether ADX confirmed a trend on that same bar. In backtesting, this looked fine. Live, it quietly broke: ADX often takes a few days to climb above the trend threshold after a price move starts. If the crossover happened while ADX was still catching up, the strategy correctly held off — and then never revisited the decision, because the “crossing” moment had already passed by the time ADX confirmed anything.

I only caught this because I was watching the live indicator values directly and noticed SMA20 sitting comfortably above SMA50, with ADX freshly past the trend threshold — every condition for a buy, and yet nothing fired. The fix was switching from event-based signals (“did the cross happen on this exact bar?”) to state-based signals (“is the market currently in a qualifying state?”) — evaluated fresh every cycle instead of only at the moment of crossing.

Re-running backtests after the fix produced a nuance worth sitting with: the fix helped one symbol (ACN, whose edge over buy-and-hold recovered almost completely) and did nothing for another (AAPL, which stayed just as underwater as before). Digging into trade counts explained why — AAPL’s extra trades came from the entry-side fix genuinely finding valid signals the old code had missed; they just hadn’t been profitable in that particular window. That’s not a lingering bug, that’s the strategy behaving exactly as designed in a regime it isn’t suited to.

Automation surfaced a second bug — a scarier one

The final incident: running unattended overnight, the bot placed fourteen buy orders instead of one. All fourteen filled.

The chain of events: the bot’s order logic only checked whether it already held a filled position before deciding to buy again — it never checked whether an order was already pending. Orders submitted outside market hours sit queued rather than filling instantly. Every 15-minute cycle, seeing no filled position yet, the bot dutifully submitted another order. When the market opened, they all filled in a stack.

Nothing catastrophic happened — this was paper money, which is exactly the point of testing with paper money — but it’s a sharp reminder that live automation finds failure modes that no amount of backtesting will ever surface, because backtests don’t have an “outside market hours” state to get confused by. The fix was straightforward once identified: check for open/pending orders, not just filled positions, before submitting anything new.

Where it stands now

The current setup:

SMA crossover gated by ADX, evaluated as a live state rather than a one-time event
ATR-scaled stop-loss instead of a flat percentage
Duplicate-order protection before every buy, sell, and stop-loss exit
Daily equity and indicator snapshots logged to CSV
A small self-updating HTML dashboard, served over Apache, comparing live strategy performance against simple buy-and-hold in real time

It’s paper-trading a real position on ACN as I write this, and I’ve got a second instance queued up to run SPY alongside it — deliberately the strategy’s weakest backtested case, because watching where something doesn’t work in real time is just as informative as watching where it does.

What I’d tell someone starting the same project

Always benchmark against doing nothing. A backtest’s return means nothing without a buy-and-hold comparison sitting right next to it.
Test every parameter somewhere it should fail, not just somewhere it should succeed. One stock, one regime, is not a test — it’s a story you’re telling yourself.
Risk parameters need to respect the asset’s own behavior. A flat stop-loss is really a bet that volatility is uniform across everything you’ll ever trade. It isn’t.
Backtests and live systems fail differently. Anything involving timing, order state, or “what happens between cycles” won’t show up until you actually run the thing continuously, unattended, for real.
A bug you catch on paper money is a gift. Fourteen unintended orders taught me more about order-state management than a design review ever would have — and cost nothing to learn.

If you’re building something similar, my honest advice is to spend less time trying to find the perfect strategy and more time building the scaffolding to honestly evaluate whatever strategy you pick. The strategy in this project is still, by design, a fairly simple one. Almost everything valuable I learned came from the process of rigorously testing it, not from the cleverness of the logic itself.

Up next: Part 2 walks through the actual setup on a Linux server, step by step, with the full working code for every file described above. Part 3 covers building the live HTML dashboard mentioned earlier, served over Apache.