come up with a growth framework

[chadwhitacre]

Started as: "go through bullseye exercise." Scope expanded to be about coming up with a growth framework generally.

Bullseye

http://tractionbook.com/bullseye.pdf

Inner Circle

Promising

Long-shot

[chadwhitacre]

Raw list from http://www.amazon.com/-/dp/B00N06Y2DW:

Viral Marketing
Public Relations (PR)
Unconventional PR
Search Engine Marketing (SEM)
Social and Display Ads
Offline Ads
Search Engine Optimization (SEO)
Content Marketing
Email Marketing
Engineering as Marketing
Target Market Blogs
Business Development (BD)
Sales
Affiliate Programs
Existing Platforms
Trade Shows
Offline Events
Speaking Engagements
Community Building

[rohitpaulk]

Content based marketing - blog post featuring success stories. Teams who are funded on gittip, individuals who make a decent amount. Write a bit about what they do, helps them get more popular, and proves that Gratipay works - so good for us too.

[chadwhitacre]

IRC

[chadwhitacre]

Some blog stats (doesn't include blog.gittip.com).

[chadwhitacre]

"New Economy"

http://neweconomy.net/
http://www.neweconomyweek.org/
http://sacred-economics.com/
http://www.garalperovitz.com/
http://charleseisenstein.net/
http://www.shareable.net/

IRC IRC

http://www.shareable.net/blog/can-new-work-really-work
http://reimaginingwork.org/
http://www.shareable.net/blog/living-the-new-economy-convergence-draws-passionate-crowd
http://www.bayareaneweconomy.org/
http://newworknewculture.org/

IRC

[chadwhitacre]

Philanthropy

http://suegardner.org/2013/10/20/whats-really-wrong-with-nonprofits-and-how-we-can-fix-it/
http://www.ssireview.org/articles/entry/how_nonprofits_get_really_big

[chadwhitacre]

That shows the percentage of users who were in fact active for the given number of weeks, out of those who could possibly have been active for the given number of weeks.

Context is https://www.youtube.com/watch?v=n_yHZ_vKjno.

IRC

retention.py

[rohitpaulk]

Although our common terminology for active users refers to those who either receive or give tips in a week, I don't think that it's the best criteria to consider for a retention chart. Since we're just taking tips, someone who created an account long ago and has been receiving $0.01 every week would be considered an active user throughout the 122 weeks.

I think a better way to look at activeness would be - people who have done either one of the following within the previous month/week/whatever-time-period-we're-using

1. People who have set a tip up to someone else, or changed an existing tip
2. People who have received new tips or whose tips have been changed (the people on the other side of point 1
3. People who have changed their takes in a team.

There are people who would fall through the cracks on this one, but I think they'd be far less than the number of non-active users we're counting in the current method. Here's a list of people I can think of who should be, but won't be included in this chart.

1. People who maintain a constant take from the team(s) they're a part of, and don't experience a change in receiving/giving often.
2. People who only give a significant amount of money, and don't change it often. Could be organizational givers, for example.

[rohitpaulk]

cc: @webmaven

[chadwhitacre]

@rohitpaulk Can you repost your comment over on #96? I think we should move the retention conversation over there to avoid overloading this ticket.

[chadwhitacre]

https://www.youtube.com/watch?v=raIUQP71SBU&t=9m16s (transcript)

Similarly my biggest fear was we spam our users and we trick them and it will alienate these people. You won't see it today but you'll see it in three years from now or four years from now, and it accelerates when you compound that with a competitor who actually builds a better product that doesn't alienate people. The most important thing that we did against our framework was I teased out virality and said you cannot do it. Don't talk about it. Don't touch it. I don't want you to give me any product plans that revolve around this idea of virality. I don’t want to hear it.

What I want to hear about is the three most difficult and hard problems that any consumer product has to deal with. How do you get people in the front door? How do you get them to an “Aha” moment as quickly as possible? And then how do you deliver core product value as often as possible? After all of that is said and done only then can you propose to me how you are going to get people to get more people. That single decision about not even allowing the conversation to revolve around this last thing in my opinion was the most important thing that we did.

When I look again in the landscape, things that scale understand that principle, whether it's explicitly or intuitively, and things that don't and also things that have this amazingly steep rise and then fall off a cliff and there are really visible examples of that today also ignore that principle. It’s the discipline to not optimize for the thing that gives you the shortest and most immediate ROI because that is never the sustainable thing that allows you to build something useful.

[chadwhitacre]

Three-part virality framework:

• payload—number of people cc'd on each message
• frequency—how often a message is sent
• conversion rate—how many people reached follow through

Re: PayPal: "I can't think of a higher conversion rate. Frequency was low. Payload was low." Discussion starts at https://www.youtube.com/watch?v=n_yHZ_vKjno&t=28m38s. Screenshot (first column is Hotmail):

[chadwhitacre]

You have to work backwards from, "What is the thing that people are here to do? What is the 'Aha!' moment that they want? Why can I not give that to them as fast as possible?" Measuring that in days is unrealistic. Measuring it in hours is unrealistic. Measuring it in minutes is ... necessary, but not sufficient. But, like, how do you get that to seconds? How do you get that to hundreds of milliseconds? That's how you win.

https://www.youtube.com/watch?v=raIUQP71SBU&t=25m01s

[chadwhitacre]

Chamath and his team started measuring things at Facebook as a way of "invalidating lore," and in the process noticed patterns of engagement—some groups of users were highly engaged while others weren't—and over time they developed an "axiom": Seven users in 10 days (given in Alex's talk as: 10 users in 14 days).

If I could give you a framework, it would probably be: You've got to start with a broad cross-section of engaged users, and when you work backwards from each of those, and you are smart enough and clever enough to really figure out the different pathways in which they got to that place, you can probably tease out what those simple things are [namely, the "axiom"], and then hopefully—and I think most products are structured this way—you can then sort of path people, more and more of those people, into those same clickflows that allow them to get to that state.

https://www.youtube.com/watch?v=raIUQP71SBU&t=34m21s

[chadwhitacre]

The slope is the company's growth rate. If there's one number every founder should always know, it's the company's growth rate. That's the measure of a startup. If you don't know that number, you don't even know if you're doing well or badly.

[...]

The best thing to measure the growth rate of is revenue. The next best, for startups that aren't charging initially, is active users.

http://www.paulgraham.com/growth.html

[chadwhitacre]

Flip-flop. Let's keep the conversation here.

More retention resources:

https://mixpanel.com/retention/
https://mixpanel.com/help/questions/articles/how-is-retention-calculated
https://keen.io/blog/47823687779/how-to-do-a-retention-analysis/
https://www.periscope.io/blog/how-to-calculate-cohort-retention-in-sql.html
http://andrewchen.co/2014/06/03/retention-is-king/
https://www.facebook.com/photo.php?v=3707283286197

[chadwhitacre]

@rohitpaulk You raise some good points.

As I mentioned in "Growing Gratipay," the delight of finding out that you've been receiving all along could be considered an 'Aha' moment for us. How does that factor into our thinking about user activity? Do we need an explicit "passive user" category for unengaged users that are in fact receiving a trickle?

What other 'Aha!' moments do we have? I think the "Who inspires you?" question prompts an aha moment: the expansive, optimistic feeling of recalling to mind someone that you find genuinely inspirational.

The "noticing the trickle" moment is the flip side of that coin:

not going to lie, being someone's answer to "who inspires you?" feels awesome. (src)

Maybe we call them "sleepers"? People who have an aha moment waiting for them but they don't know it yet. Interestingly, this flips the quote from Chamath above: "Measuring that in days is unrealistic. Measuring it in hours is unrealistic. Measuring it in minutes is ... necessary, but not sufficient. But, like, how do you get that to seconds? How do you get that to hundreds of milliseconds? That's how you win." One way that Gratipay functions is that the 'aha' moment can take months or years to manifest. That's part of what makes it so powerful, I think.

[chadwhitacre]

Gratitude and generosity. Those are our two aha moments. Gratitude the first time you receive. Generosity the first time you give.

[chadwhitacre]

Here's a list of people I can think of who should be, but won't be included in this chart.

If we include exchanges in activity, then I think we'd cover the two cases you mention. People who are depending on Gratipay for income are going to be withdrawing to their bank account every week. That is a central activity that we want to encourage. Same with patrons: they'll have their credit card charged every week.

Part of the twist for Gratipay is that we're "set it and forget it" by design. The whole point is to lay a foundation for people's lives economically that they don't have to think about constantly. The point of Gratipay is to free people up to spend time on more important things. If someone is making a living on Gratipay as a member of a team or three, we don't need them to be clicking on the site constantly. Click elsewhere!

[webmaven]

@whit537 so, do you comprehend measuring retention by cohort now?

[chadwhitacre]

@webmaven No. I closed that ticket because I decided to keep the conversation in one place.

[webmaven]

OK, not deeply comprehend (that will take a long time), but do you understand dividing people into cohorts, and do you understand what the chart should display in terms of retention over time for each cohort (whatever definition of activity you choose to use)?

Enough to rewrite the script you had, anyway?

[chadwhitacre]

For the record here's my broken attempt at doing the right thing with cohorts:

[webmaven]

That still looks the same as last time...

[rohitpaulk]

If we include exchanges in activity, then I think we'd cover the two cases you mention. People who are depending on Gratipay for income are going to be withdrawing to their bank account every week. That is a central activity that we want to encourage. Same with patrons: they'll have their credit card charged every week.

I agree. If they're withdrawing or being charged, they're certainly aware of their actions on Gratipay and hence can be termed as active.

[webmaven]

So, aside from measuring "% of active users over time per cohort", another interesting chart would be "average # of weeks from signup to activation (ie. first transaction) per cohort". For me personally, I joined (created an account) ~3 months before I made a first transaction. I have some ideas around lowering that # (if it needs to be lowered), but first we need to see the data.

[chadwhitacre]

@webmaven Here is the chart from the video about growth from the Stanford class that I used as a model to generate the chart above:

He says "it'll get noisy out towards the right-hand side" (here's a transcript of Alex's talk, btw), and so I think ours looks not dissimilar from his idealized example. Here's ours again:

Alex also says, "normalize on a cohort basis," which I took to mean that one wants to show the percentage of retained users out of the total number of users that could possibly have been retained that long at each point in time, rather than the absolute number of users retained at each point in time. He actually overlays the absolute number of users later in the talk:

For us, there's only one cohort that could possibly be part of the data point at 124 weeks: only people who joined the first week could possibly have stuck around the whole time. And for 123, there are two cohorts that could be part of that data point: people that joined in week one, and people that joined in week two. At 122 weeks, there are three cohorts, etc.

In retention.py we are generating a distribution of percentages for each cohort, and then we're folding those distributions into one by averaging each point on the x axis: for "0 weeks from first activity" we have 124 percentages to average together. For "50 weeks from first activity" we have 74 data points to average together, and for "124 weeks from first activity" we have only one data point: people who joined in the very first week.

This bar chart from a blog post on retention analysis by Woopra gives an example of an app with much less retention than ours:

[chadwhitacre]

[webmaven]

@whit537 Did we get the CSV?

[chadwhitacre]

Did we get the CSV?

Nope, not yet. I just sent a follow-up email.

[chadwhitacre]

Found some good practical advice on working with Data and A/B Test: "The Agony and Ecstasy of Building with Data" (more Facebook).

[webmaven]

👍

[chadwhitacre]

Another tool: https://qualaroo.com/.

[chadwhitacre]

Funnel (copied from #101 (comment)):

1. be interested in what we have to offer
2. hear about us
3. visit
4. read about
5. explore examples (find someone to give to?)
6. sign up
7. enter credit card info
8. find someone to give to (if haven't yet)
9. pick an amount

[chadwhitacre]

IRC

[chadwhitacre]

[chrisdev]

@webmaven @whit537 to convert read SPSS files use:

pip install savReaderWriter --allow-all-external

It worked for me a few years ago the IBM guys are keeping it up to date so it should work.
You get back a list use csv writer or pandas to convert to CSV

https://pypi.python.org/pypi/savReaderWriter

[chadwhitacre]

Good find, @chrisdev!

[chadwhitacre]

Another tack: What is the pain point we address?

Patreon addresses a crystal-clear pain point:

Patreon is not for people starting making content, it's not for people with no fans. Patreon is for people on the cusp of making a living, who have hundreds of thousands of fans. It's for people who are micro-celebrities, and for people who are making stuff and having tremendous success, and are still working at Starbucks for ten dollars an hour, two days a week, because they can't afford to do it with just their ad revenue.

In thinking through how we're different from Patreon, it strikes me that we don't have nearly so crisp an answer.

[chadwhitacre]

Speaking for myself, Gratipay addresses the pain point of wanting to be a productive contributor to society but not being able to stomach working for a closed company.

[Changaco]

Gratipay tries to address the pain point of sending and receiving recurrent payments easily without paying extra fees. The original use case was funding free software, and that generalizes to funding the commons (i.e. non-rival goods that are free to use), but really Gratipay is a neutral tool, like the Internet, and the beauty of such a tool is that it can have lots of uses its creators hadn't thought of.

[chadwhitacre]

"Teams Work" and IRC

[Changaco]

Reworded:

The original pain point was not being able to work on free software. That generalizes to not being able to work on improving the commons, i.e. non-rival goods that are free to use, because many business models don't apply to them, so it's difficult to fund that kind of work.

The way Gratipay tries to address this pain point is by providing a service that allows people to send and receive recurrent payments easily, so they can fund their work with donations.

But funding the commons is not the only use case for a payment service that doesn't have fees, and that's why you can't find the pain point that Gratipay addresses, because there are more than one.

[chadwhitacre]

IRC

[chadwhitacre]

Sent in private email:

Scott,

Been thinkin'.

Gratipay is a product for entrepreneurs who are building the next Wikipedia and need a payments & payroll platform. A relationship with an accelerator could be mutually beneficial since they would represent so many entrepreneurs, some of whom might be looking for Gratipay. :-)

Any chance you'd be willing to share Dave Blanchard's email address with me?

chad

[chadwhitacre]

Here's an organization that seems like a good fit for Gratipay:

http://www.therealjunkfoodproject.co.uk/

I just don't feel good about cranking up our sales engine right now when our product is so bad. :-(

[chadwhitacre]

Like, if I had more confidence in our product, I would compile a list of as many TRJFPs as I could, and I would contact them methodically.

[chadwhitacre]

1. We've had good luck with in-person marketing channels (user groups, conferences).
2. We want to reach non-geeks.
3. ∴ Let's try shopping malls!

[chadwhitacre]

Also, truck (#172) and pop-up store (#173).

[chadwhitacre]

Music festivals: #174.

[chadwhitacre]

The growth framework is:

1. Build a kick-ass v1 product.
2. Spread the word via in-person marketing channels.

I'm glad we got our feet wet with formal A/B testing and metric-driven development. !m @webmaven. I want to revisit those tools after v1 is out and we have another order or two of magnitude in scale.

[webmaven]

Just note that you should be A/B testing your in-person marketing efforts too.

[chadwhitacre]

Filing this here, at the risk of stirring the pot ... 🍯

Stepping away from A/B testing does not mean we have abandoned efforts to improve conversion. Rather we are improving our conversion through other methods (yes, A/B testing isn't your only option).

https://www.stickermule.com/blog/hidden-costs-of-AB-testing

[webmaven]

Pot stirring is what some of us do best! 🥄🌪️🍯