Hacker News: conductrics

CUPED's Sting: More Power More Underpowered A/B Tests

conductrics — Fri, 11 Apr 2025 19:30:02 +0000

Article URL: https://blog.conductrics.com/cupeds-sting-more-power-more-underpowered-a-b-tests/

Comments URL: https://news.ycombinator.com/item?id=43657563

Points: 1

# Comments: 0

New comment by conductrics in "Simple Zero Analysis Multi-Armed Bandit"

conductrics — Sun, 06 Apr 2025 13:11:23 +0000

A simple Zero analysis Multi-Armed Bandit Reproposing Pearson-Neyman sample size calculations to create simple bandits with no need for dedicated software. Obviously not optimal but often a very good option.

Simple Zero Analysis Multi-Armed Bandit

conductrics — Sun, 06 Apr 2025 13:11:23 +0000

Article URL: https://blog.conductrics.com/adjusted-power-multi-armed-bandit/

Comments URL: https://news.ycombinator.com/item?id=43601201

Points: 1

# Comments: 1

Explaining Concepts Behind AB Testing Without Math

conductrics — Wed, 01 Jun 2022 17:01:32 +0000

Article URL: https://conductrics.com/ab-testing-ruling-out-to-conclude/

Comments URL: https://news.ycombinator.com/item?id=31584789

Points: 2

# Comments: 0

New comment by conductrics in "Optimizely to be acquired by Episerver"

conductrics — Fri, 04 Sep 2020 00:04:23 +0000

Are you suggesting that revenues fell after bailing on SMB. That is hard to imagine, but I guess possible. Given that the core of the industry is about asking counterfactual questions, I would think the appropriate question would be 'would they be more valuable now if they had not gone to the enterprise - would they even have the deal they did wind up getting?'rather than are they more valuable now then they were 4 years ago. Hard to know, but my guess is that they wouldn't. A simple web editor with a random number generator isn't going to be of interest to anyone looking to buy. The larger problem is that statistical inference is hard - it just is. And, unlike analytics, where you are just placing sensors into an existing system, here you need to also place actuators, so implementation is much more complicated. That means that at any scale, both the marketing team, and the dev/IT teams need to be involved for anyone to get value and not wind up breaking systems all the time. Software like theirs, and ours, isn't magic, and without good editorial and hard work by the client, the entire exercise is more statistical theater than science. And that material fact was always in conflict with the rhetoric that they make AB Testing easy for everyone. It lends itself to a particular type of solutionism that VCs and the larger industry are prone to be seduced by. Self service to SMB clients might be a profitable biz, but perhaps not enough to service such a large amount of venture funding. FWIW VWO also tries compete at the enterprise.

A/B Testing Tools in 2020

conductrics — Tue, 03 Dec 2019 02:50:33 +0000

Article URL: https://www.alexbirkett.com/best-ab-testing-tools/

Comments URL: https://news.ycombinator.com/item?id=21689482

Points: 2

# Comments: 0

Intro to Entropy and Information Theory for Digital Analysts [video]

conductrics — Mon, 08 Jul 2019 15:09:11 +0000

Article URL: https://www.youtube.com/watch?v=sMTdNKYEVSk

Comments URL: https://news.ycombinator.com/item?id=20383374

Points: 3

# Comments: 0

Do No Harm or AB Testing Without P-Values

conductrics — Wed, 15 Aug 2018 18:45:51 +0000

Article URL: https://conductrics.com/do-no-harm-or-ab-testing-without-p-values/

Comments URL: https://news.ycombinator.com/item?id=17768617

Points: 2

# Comments: 0

Do No Harm or AB Testing Without Using P-Values

conductrics — Mon, 02 Apr 2018 11:23:15 +0000

Article URL: https://conductrics.com/do-no-harm-or-ab-testing-without-p-values/

Comments URL: https://news.ycombinator.com/item?id=16733660

Points: 1

# Comments: 0

Thompson Sampling or how I learned to love Roulette

conductrics — Fri, 12 Jan 2018 16:19:53 +0000

Article URL: https://conductrics.com/lets-play-roulette-using-machine-learning-and-thompson-sampling-for-digital-optimization/

Comments URL: https://news.ycombinator.com/item?id=16133982

Points: 1

# Comments: 0

Reinforcement Learning: Going from AB Testing to AI

conductrics — Wed, 14 Jun 2017 19:45:22 +0000

Article URL: https://www.youtube.com/watch?v=ZcYwNItzb4Y

Comments URL: https://news.ycombinator.com/item?id=14555479

Points: 1

# Comments: 0

Hacking AB Testing: Reinforcement Learning – Going from AB Testing to AI

conductrics — Mon, 17 Apr 2017 16:16:29 +0000

Article URL: https://www.youtube.com/watch?v=ZcYwNItzb4Y

Comments URL: https://news.ycombinator.com/item?id=14132051

Points: 2

# Comments: 0

New comment by conductrics in "20 lines of code that beat A/B testing (2012)"

conductrics — Wed, 06 Apr 2016 14:55:31 +0000

When thinking about what type of approach is best, first think about the nature of the problem. First is it a real optimization problem, IOW are you more concerned with learning an optimal controller for your marketing application? If so then ask: 1) Is the problem/information perishable - for example Perishable: picking headlines for News articles; Not Perishable: Site redesign. If Perishable then Bandit might give you real returns. 2) Complexity: Are you using covariates (contextual bandits, Reinforcement learning with function approximation) or not. If you are, then you might want your targeting model to serve up best the predicted options in subspaces (frequent user types) that it has more experiences in and for it to explore more in less frequently visited areas (less common user types). 3) Scale/Automation: You have tons of transactional decision problems, and it just doesn't scale to have people running many AB Tests.

Often it is a mix - you might use a bandit approach with your predictive targeting, but you also should A/B tests the impact of your targeting model approach vs a current default and/or a random draw. see slides 59-65: http://www.slideshare.net/mgershoff/predictive-analytics-bro...

For a quick bandit overview check out: http://www.slideshare.net/mgershoff/conductrics-bandit-basic...

When to Run Bandit Tests Instead of A/B/n Tests

conductrics — Thu, 17 Sep 2015 19:36:34 +0000

Article URL: http://conversionxl.com/bandit-tests/

Comments URL: https://news.ycombinator.com/item?id=10235561

Points: 6

# Comments: 0

Top 7 Data Scientists before there was Data Science

conductrics — Thu, 17 Sep 2015 19:34:14 +0000

Article URL: http://conductrics.com/the-worlds-7-top-data-scientists-before-there-was-datascience/

Comments URL: https://news.ycombinator.com/item?id=10235551

Points: 3

# Comments: 0

Big Data Is Actually About the Very Small

conductrics — Tue, 09 Dec 2014 06:44:23 +0000

Article URL: http://conductrics.com/big-data-is-actually-about-the-very-small/

Comments URL: https://news.ycombinator.com/item?id=8721385

Points: 1

# Comments: 0

Pulling back the curtain on p-values or How I learned to love small data

conductrics — Wed, 21 May 2014 12:41:06 +0000

Article URL: http://conversionxl.com/pulling-back-curtain-p-values-learned-love-small-data

Comments URL: https://news.ycombinator.com/item?id=7777876

Points: 4

# Comments: 0

New comment by conductrics in "Ask HN: Is "Gödel, Escher, Bach" still worth reading?"

conductrics — Sat, 11 Jan 2014 01:00:40 +0000

I read GEB as my subway book, back in 1999 (during a work stint in Paris), during my daily commute to work. Years later, when I was looking for a change, I started thinking about GEB. Even though I had no CS background,I decided to apply to a few AI programs and wound up going to the University of Edinburgh for an Msc. in AI in '05. Now I am a co-founder of software start up that applies reinforcement learning, a method from ai -lower case ;-) to conversion optimization. Obviously, I can't say if you will find it a worthwhile read, but I do look back on it as a significant influence on a major pivot in my life.

The World’s Top 7 Data Scientists before there was Data Science

conductrics — Sun, 03 Nov 2013 20:36:32 +0000

Article URL: http://conductrics.com/the-worlds-7-top-data-scientists-before-there-was-datascience/

Hacker News: conductrics

CUPED's Sting: More Power More Underpowered A/B Tests

New comment by conductrics in "Simple Zero Analysis Multi-Armed Bandit"

Simple Zero Analysis Multi-Armed Bandit

Explaining Concepts Behind AB Testing Without Math

New comment by conductrics in "Optimizely to be acquired by Episerver"

A/B Testing Tools in 2020

Intro to Entropy and Information Theory for Digital Analysts [video]

Do No Harm or AB Testing Without P-Values

Do No Harm or AB Testing Without Using P-Values

Thompson Sampling or how I learned to love Roulette

Reinforcement Learning: Going from AB Testing to AI

Hacking AB Testing: Reinforcement Learning – Going from AB Testing to AI

New comment by conductrics in "20 lines of code that beat A/B testing (2012)"

When to Run Bandit Tests Instead of A/B/n Tests

Top 7 Data Scientists before there was Data Science

Big Data Is Actually About the Very Small

Pulling back the curtain on p-values or How I learned to love small data

New comment by conductrics in "Ask HN: Is "Gödel, Escher, Bach" still worth reading?"

The World’s Top 7 Data Scientists before there was Data Science

Unfulfilled Promise: Thoughts on the Promises API