A fluent answer is still an unchecked test

Partly: ChatGPT can turn written rules into code, and some versions can run it on a data file you upload. What it can't do is promise the data is right, or that the code never peeks at the future.

The biggest trap is asking for results without code. A chat model can describe a plausible win rate and equity curve in plain text. If no code ran on real data, those numbers came from nowhere.

So treat it as a fast junior programmer. It writes quickly and makes confident mistakes. You stay responsible for the data going in and for checking what comes out.

Expect it to forget your rules between chats unless you paste them in again. Save the rule sheet and the exact prompt you used. Then the next run tests the same strategy, and you can compare the two honestly.

The rest of this piece covers what to hand over, then how to check what comes back.

Hand it the translation work. Describe the rules in plain words and ask for Python or Pine Script. It'll produce something runnable faster than you could type it.

It's also good at finding holes. Ask it what your rules leave undefined. A decent answer raises questions like these. What happens if the stop and target are both hit inside the same bar? Do you enter at the signal bar's close or the next bar's open? What if price gaps past your stop overnight?

Those questions are worth more than the code. Each one changes the result, and a vague rule sheet hides all of them.

Keep the prompt specific. Give the timeframe, the symbol list, the cost per trade and the exact entry and exit conditions. Ask for a list of individual trades as well as the summary.

A test is only as good as the price history under it. The model has no reliable way to know if yours is clean.

Check for missing bars, duplicated bars and odd spikes from bad prints. Check whether prices are adjusted for splits. An unadjusted split looks like a crash, and a short strategy will look brilliant on it. For holds of weeks or months, check whether dividends are in the prices too.

Then ask what's missing entirely. A list of today's index members leaves out companies that were dropped or went bust. Testing only the survivors tends to make long strategies look better than they were.

Time zones do their damage in silence. If the file is in UTC and your rules say 10:00, the code may trade the wrong hour every day. None of this shows up in a neat summary table. Look at the raw file yourself before any code touches it.

Look-ahead bias means the code uses information that didn't exist yet when the trade was taken. It's a classic way for a test to print money it could never have made.

The textbook version: the signal uses today's close, but the trade enters at today's open. On paper you bought before the information arrived. In pandas code, the fix is often a missing shift by one bar.

Another version uses a daily high or low to decide an intraday entry. You only know the day's high once the day is over.

A subtler version sits in the exit logic. When stop and target both fall inside one bar, the code may assume the target filled first. Rerun it assuming the stop filled first, and see how much the result drops.

A model can write this bug and then fail to spot it, because the code runs without errors. Ask it directly where look-ahead could creep in. Then check those lines yourself, since a wrong answer sounds exactly like a right one.

Never accept a result you can't trace to code and data. Ask for the code, run it yourself, and have it print every trade with timestamps.

Pick five trades at random from that list. Open each one on a chart and find the bar where the signal fired. Was the condition really true using only data up to that bar? Did the entry price exist at that moment?

Then move the main setting a notch either side. If the result collapses when 20 becomes 18 or 22, the test is fitted to noise.

Finally, add costs. Ask for a version with commission and a tick or two of slippage per side. A strategy that only works at zero cost won't work in an account. Once it passes all of this, trade it small and compare. Keep the code and the data file together in a dated folder. That way you can rerun the exact test months later.

Run through all ten before you trust a single number.