What's my next moves here?

1. Analyse the conversion funnel data with a larger sample size (Perhaps target 1000 sessions or at least 10 successful conversions from “strangers”) If not, it’s gonna hard to make sense of the data above and too risky to interpret “emerging trends” like something is broken or working with a small sample size

So I should continue running the ad?

If that's the most effective way for you to get to say ~1k sessions, then yes, continue

I agree with @Randy Chai the sample size is way too small for you to come to a concrete conclusion.

Marketing neophyte here (engineering background). While I agree that getting more data can result in more statistically significant results, is that necessary in this case? We can see from the data that of 34 users who reached checkout, only 2 converted. 2 out of a sample size of 34 seems to clearly point to something wrong in the checkout flow, no? Is more data going to be useful?

How can one be confident that 2/34 conversations = “something wrong in the checkout flow”? What if 32/34 of the visitors aren’t the ideal target audience of this brand? As if that (wrong type of audience visiting) is indeed the case, the site owner will spend resources to fix his/her site’s checkout flow and land on the same issue after X period of testing. Or even simply that 32/34 are “window shoppers”, not ready to buy today? Imo, it’s not ideal or wise to make a decision or come to a conclusion based on 34 site sessions (unless you’re selling luxury goods, property, high ticket items)

“What if 32/34 of the visitors aren’t the ideal target audience of this brand?” If that’s the case, how does increasing the sample size help? For example, say they get to 10,000 added to cart, and 800 reached checkout. Does that additional accuracy help to diagnose the problem? (trying in good faith to understand here)

It doesn't, not directly at least. But it mitigates the “window shoppers” risk

In my experience with fashion marketing, the cart abandonment rate is quite big. It’s common knowledge that about 7 out of every 10 shoppers won’t complete their transaction, but I think on average, it’s even bigger in the fashion sector. However I do have a few points to add here. 1. The sample is too small, but you can ry to figure out the cause without spending more money. It’s possible that you have an issue in the checkout flow that you might not be aware of. I’ve found that the best way to find out the issue is by taking the buyer jounrey my self, and asking a few friends to do it because as much as I try I have biased eyes. Sometimes it’s not even an issue, rather a customer experience hurdle (the sipping cost was bigger than they expected, etc). 2. The target might be the problem - but it also might be a case of user readiness, what time of the month it is (and no, I’m not referring to periods, rather salary payout dates) and the product in questions (the more expensive the product, the longer the evaluation period, especially from un unknown brand). Also, we are on the onset of a recession so it’s normal for buyers to evaluate their non-essential purchases more carefully. Just be aware that sometimes replicating the success of a previous campaign might be hard due to factors that have nothing to do with he campaign. 3. My advice would be to check out the user journey and compare data you have from previous campaigns, having the other factors in mind. And if you haven’t already, set up a 15% off your first purchase popup to capture emails and boost those returning visits as well as your brand awareness. Hope this helps.

Just curious to know is this screenshot from shopify?

whats your niche?