Michelle Sun San Francisco

// 6 Essential Questions to Ask Yourself before Running that A/B Test//

image

Thanks to the many tools out there today for marketing and product analytics, running tests effectively is easier. At Buffer, we are currently in a good swing of running experiments. When running experiments, I have found it helpful to keep the following questions in mind, as a framework in iterating from asking questions to gaining insights from experiments.

(1) Which top metric are we trying to improve? 

Swimming through a sea of data, it is easy to get lost in the details and dig for the sake of more data. I find it essential to zoom out, and start with the key metrics. There are different frameworks out there, like Dave McClure’s Pirate Metrics (AARRR), Eric Ries’ Engines of Growth, and Sean Ellis’ Startup Growth Pyramid. I believe they serve the same purpose of structuring your thought process, so it’s more important to pick one and stick with it.

For this example, we pick Pirate Metrics’ framework and Acquisition as the top metric.

(2) What can you anchor your metric against? 

Once the key metric is picked, identify the mismatch between your expectation and reality. A common question for someone new to metrics is, how do you form those expectations? One way is to base them on industry averages. A useful reference is the plethora of answers on Quora. Eg. for landing page conversion, where 5-15% is what considered the reasonable range. If your conversion rate is on the lower end of the range, your question may be, “Why is the landing page conversion at 5.1% but not 10%?”. 

Another way to anchor expectations is to segment your metrics into more dimensions. For landing page, look at the overall average, and check the individual assumptions against that average. Then your question may be “Why are visitors from Lifehacker.com converting at 6% but those from Feedly at 10%?”.

For this example, in this case, our question will be “Why is the landing page conversion at 5.1% but not 10%?” 

(3) What is your hypothesis about why this metric is at x%? 

Once you identify that gap, think about a list of reasons why you think may be the cause of this mismatch. For a low-converting landing page, write out a list of hypotheses. For example: 

- Sign up button is not conspicuous enough 

- Too many buttons on the page confuse the user

- Value proposition is not clear

- Page load is too slow

- The page is not mobile optimized

The list goes on.

For this example, we choose the hypothesis that “Value proposition is not clear enough”.

(4) How do we improve this metric? 

This is the most fun part where you brainstorm the list of experiments, ie, potential changes that you can make to the page to improve the metric in question. 

With our hypothesis of the value proposition being not clear enough, here is a sample list of potential reasons behind this mismatch. 

a) Cut the copy to shorter sentences

b) Change from paragraph to bullet points

c) Include a video demo

d) Include screenshots

e) Try moving social proof above the fold

Let’s say we pick a), to cut the copy to shorter sentences as our experiment. 

(5) What do we expect from this experiment? 

This is the trickiest part when starting to run experiments. In the beginning, setting an expectation on results may be a shot in the dark. Don’t let the uncertainty of results deter you. After a few rounds of comparing expected and actual results, you can tune the intuition of future experiments better - ie, improve the intuition of how users react to changes to different page elements. 

In our example, let’s say our intuition is that shorter sentences improve landing page conversion. But how much? How do I know how much this may increase conversion rate by?

Eg, we note down that we expect the shorter copy will improve conversion by 2 percentage point, so from 5.1% to around 7.1%. 

(6) What was the result and what did we learn from it? 

When results are positive (“conversions went up from 5.1% to 8%!”), or results did not turn out as expected (“conversion did not move”), take a moment to review and develop a hypothesis on why that is. Sometimes, unexpected results from experiments can teach us even more than those that are expected. 

E.g, our shortened copy on the landing page did not move the needle, perhaps it’s the positioning of the copy that needs to be changed. Or, it’s the sign-up button that needs more work.  

The purpose of A/B testing is not only to improve specific metrics, but also derive insights to inform future experiments. 

What other questions or areas do you think about when running an experiment? How do you structure the process? I’d love to hear from you! 

Special thanks to Leo Widrich for reading through the draft and providing helpful comments. 

// Lessons learned so far from our transition to growth at Buffer //

image

Sean Ellis detailed at the Growth Hackers Conference on three key stages of growth(1).  With his experience working with various startups including Dropbox and Eventbrite, he coined a concept “Growth Pyramid”, from product market fit, then transition to growth, and eventually, growth.   

At Buffer, in recent months, we have been focused on the transition to growth.  This post intends to share some key lessons learned so far.  I hope it will be useful for other startups at a similar stage. 

Growth hacking is what you do only after you have growth. You need to prove out that you have a core value prop, an asset that people like and use. ” 

- Keith Rabois, former COO at Square  


 

(1) Define and refine product goals

"What is the core value that the product delivers?  What is the wow experience (or "aha" moment) that we want the user to experience?"

The foundation of growth is delivering real value.  Hopefully these are obvious questions that you can readily answer.  In all, it goes back to what and whose problem are you solving.  The answers should be short and simple, and can be explained to someone that is not in the domain / industry.  I find it helpful to go talk to a few friends who are not in startups, and refine after each pitch until they understand right away in one description. 

(2) Understand existing behavior

"How are people using our product right now? How are previous level of engagement leading to subsequent usage?" 

Before starting to actively track metrics, you need to know what metrics should you be tracking.  That is something we come across when we were about to make a dashboard for internal reference.  

Exploratory analysis is useful in uncovering usage patterns, similar to Facebook’s 7 Friends in 10 Days.  Andrew Chen has written up a great blog post on how to get those insights for your own startup. 

At Buffer what helped us got up to speed was whipping up a simple excel spreadsheet.  Choose any tool that is handiest for you at this stage; the goal is to make sense of the numbers quickly.  At the end of this step, you should know your company’s funnel metrics and current trends for each off top of your mind.  

(3) Segment and implement user analytics

"Which users are most active? Eg, for users that signed up on iPhone or Android, how is their behavior different?" 

Once you have a fundamental idea of how the macro level product usage looks like, it is time to look beyond aggregate data.  Segment data to answer more specific questions.  For example, when segmenting the activation rate of signups from different channels, we discovered that activation rate on our web dashboard almost halved with the re-design back in December (while iPhone activation shot up!).

 image

This is also a good time to implement user analytics.  Storing events for each user systematically is an investment that pays off.  For example, if you’d like to answer whether users who have received a certain email newsletter are more likely to upgrade or not, user analytics provide a straightforward way to query against an existing, nicely structured database. 

(4) Start running experiments

Velocity and data can change the company culture.  You don’t have to finish all these analysis to start running experiments.  It is a virtuous cycle where experiments can drive more understanding on the existing data as well.  We have seen speed of rolling out experiments picking up since we started to have at least one experiment running at a given point in time.  

 

Other things to watch out

 

Keep track of all questions and experiments.  Exploratory analysis involves looking into user behavior and running different types of analysis.  By definition, some of these analyses may yield interesting insights and most do not.  Don’t despair; keep a log of questions the team asked and the results for each analysis.  Often, hypotheses change in a product, user behavior can evolve and it is useful to reference back past analyses and experiments.  

Set a deadline for exploratory analysis.  There is always another way to slice the data. There is always a better way to build a churn prediction model.  Start running experiments anyway!  When transitioning to growth, think of these analyses more of hypotheses that should be revisited regularly. 

 

Is your startup also at the stage of transition to growth?  What are some of the key lessons from your transition? 

 

(1) For more extensive notes on Growth Hacker Conference and on Sean Ellis’ presentation on Growth Pyramid, check out this blog post by Sandi Macpherson on Quibb.

Special thanks to Joel Gascoigne for reading through earlier versions of the draft. 

Background to Sean Ellis’ “stages of growth” chart, credit to Thundermark

// Making Improvement the Centerpiece //

image

WIth summer just around the corner, wedding season is kicking into full gear.  My best friend, busy preparing for her sister’s wedding, was telling me about the exquisite centerpiece in great enthusiasm.  A centerpiece, I quickly learned, is the most important item in the middle of the dining table.

 

On the table, a centerpiece is a large central object which serves a decorative purpose.  Centerpieces set the theme of the decorations and bring extra decorations to the room […] However, centrepieces should not be too large, to avoid difficulty with visibility around the table and to allow for the easier serving of dishes.” - Wikipedia

 

That reminded me of a word cloud last week that was circulated within our team - in the prominent center, it was not the company name, but one of the values that have been top of mind for most of us.  

image

Yes, improvement is the centerpiece in Buffer's 'dining table'. 


 

"The most important item, in the middle of the dining table."

“Without continual growth and progress, such words as improvement, achievement, and success have no meaning.” - Benjamin Franklin 

Everyday at the end of the work day, our team writes down a ‘done’ list of the day using and share it on IDoneThis. In addition to tasks, we include a section of improvement. Writing the improvement segment often requires a moment of pause and reflection. 

If there was one theme in Ben Franklin’s life, it is probably the focus on self improvement.  His daily routine has been a huge inspiration to me; it involves early rising, learning and most importantly, beginning and ending with two questions: “What good shall I do this day?” and “What good have I done today?”

Writing out improvement items each day has helped me focus on both the learning aspect and the ‘good’ I wanted to do. 

image

 

Keep it small

"A centerpiece should not obstruct the serving of dishes and visibility among guests."  Similarly, daily improvement has worked best for me when it is bite-sized.  It is easier to focus on no more than two improvements at any given day.  Being too ambitious can run into the risk of disappointing oneself.  

 

Setting the theme and bringing extra goodness to everyday routine

Like a centerpiece does to the party, improvements set the tone of the day.  Like anything related to building or changing a habit, it takes time to solidify.  Setting a theme for each week allows time for experimentation and failing.  

At the end of each week, think of one thing that can be carried over to the subsequent week.  That helps moving from actively working on an improvement, to going onto maintenance mode and working on another area of improvement.  Two weeks ago, I began the week with the intention to improve my focus.  Then throughout the week, I became aware of bad habits like keeping too many tabs open.  Wrapping up the week, I installed Fluid and Chrome Extension that limit my tabs automatically.  By the middle of the following week, even though I was not actively trying to improve on my focus, the times I hit the limit have gradually decreased to 1-2 times each day. 

image

 

Ask for help 

There is something incredibly powerful in being open about improvements.  Showing imperfection requires humility and confidence.  Once I broadcast the intention to improve within the team, the accountability helps propel me to follow through with it.  

Being vulnerable and asking for help can also open up others and build trust.  When one of our teammates brought up an improvement about stopping nail-biting, it spurred a series of confession among a few other coworkers about the same bad habit!  In the same vein, my two amazing coworkers Carolyn and Andy have suggested the two apps mentioned above that limit multiple browser tabs. 

 

Being good vs. getting better

Everybody likes to do stuff they’re good at. When we’re doing the types of tasks and projects we’ve already mastered, we feel in control and confident. But settling into our sweet spots – and avoiding new experiences that require us to “stretch” – comes with consequences. 

Getting Better vs. Being Good 

Being good involves proving you have ability and showing you know and are good at something, while getting better emphasizes on developing ability and learning to master a new skill.

In a hectic startup life, it is easy to be lost in the hustle and get consumed by work.  Finding improvement as the centerpiece has helped me set a tone and stay mindful in a busy schedule.  What is your centerpiece? 

 

Photo Credit : Centerpiece from MarthaStewart.com, Benjamin Franklin routine from DailyRoutines

// Connecting the Dots//

'You can't connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future. You have to trust in something — your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.' - Steve Jobs

As the topic of career transitions came up in a conversation, a friend from college asked, “How do you think your experience in finance is being applied to what you do now?” His question was spot on; truth is, my years of financial modeling, various methodologies of valuation, stock market analysis couldn’t seem further away from what I have been doing in the past two years in technology.  

With Buffer since earlier this year, my work has been around building out metrics from ground up.  That includes a broad range of analytics work from company level model on monthly revenue growth and cost projections, to funnel metrics and per user economics.  After putting away my excel fingers for years, I suddenly found them handy and almost second nature when I pick it back up.  Looking back, I’m starting to see the dots connecting.  

To my surprise, the years in investment banking had been silently preparing me for this role. 

A data driven mindset

Graduating with a degree in Economics from UChicago, I went into banking with a theoretical, liberal arts educational background.  The junior year internship and the first few months full time was an immersion to the mindset of thinking in data.  Eventually, on top of the excel shortcut keys ingrained into muscle memory, I also became fluent with both macro (market and industry) and micro (building company, operational models) analysis.  As we first started modeling growth at Buffer, I am thankful to have had the training of being detail-oriented and questioning assumptions with each analysis.  

Thrive in fast-paced, changing environment with a can-do spirit. 

At banking, I started off in the Tech and New Media team.  During the financial crisis with a halved headcount, I took up an additional role in the Telecoms team.  Often, projects come in seemingly impossible deadlines and each team member takes on additional tasks to meet clients’ expectations.  

Eric Schmidt once gave Sheryl Sandberg the advice, ‘Get on a rocket ship’.  In a fast growing startup, it is not uncommon that there are more job titles than people.  One of our back-end engineers, Colin, while helping out in the support inbox, came across a user who could not install Buffer’s browser extension.  In response, he sent over a customized extension to the user.  Wow. 

Building metrics at Buffer feels a lot like building a new product in that, the team is treading a lot of unchartered waters.  It is both crucial and rewarding to think outside of the box (and job description) to get things done.  

Do the right thing 

My first project at banking was the IPO of Alibaba.com, a large ecommerce site in China.  It was Goldman’s milestone deal that year, at the peak of the market.  After the company went public, my team were mandated to issue an investment recommendation for the stock.  The stock had tripled on the first day of trading, and had reached a P/E of over 100x (compared to around 30X for a growth staged internet company).  My then manager stayed true to his analysis and issued a Sell rating, advising the market against buying the stock, even though our bank was paid by Alibaba.com to run the IPO process. 

One of the things I admire about the culture at Buffer is the commitment to openness even to a point where it means to be vulnerable.  After a big mistake in my second week, I learned that our team adopts the 5 Whys methodology to learn from each other’s mistakes.  In addition, with an overall focus on customer happiness, on a busy day of important announcement, everyone on the team, including the engineers and the founders, would jump into the inbox to answer users’ questions. 


Making a career transition from finance to technology has not been a smooth path for me.  When I made the leap of faith, I definitely did not have the foresight that the dots will align like this.  One of the demons I continue to face was the economist in me, who makes decisions based out of maximizing utility functions (making an optimal choice given a set of possible options and constraints).  Over the years, I realize intuition and passion can be hard to quantify, but they helped me be resilient and persistent when times get tough.  

In closing, I would like to share another quote by Steve Jobs that never ceased to resonate in me.  

"For the past 33 years, I have looked in the mirror every morning and asked myself: "If today were the last day of my life, would I want to do what I am about to do today?" And whenever the answer has been "No" for too many days in a row, I know I need to change something…almost everything - all external expectations, all pride, all fear of embarrassment or failure - these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose."

Are you in the middle of a career transition?  Or have you made a career change, and what was your experience in connecting the dots by looking back?  I’d love to hear your stories. 

// Rethinking Our Diet//

image

So, Google reader is closing down. It feels as if a familiar restaurant in the neighborhood is going out of business, one that I have grown to like and frequent for convenience. I am one of those people that likes to go the same restaurant around my neighborhood and orders the same food that I know I like. When a restaurant closes, it abruptly ends the cycle and it is uncomfortable. On the bright side, it presents a chance to reconsider a habit, rethink my diet. 

There are three types of readers that have been feeding my information diet. A few years ago when I started following tech blogs, I relied on Google Reader, a “source-based reader" and subscribed to a long list of blogs, such as Techcrunch, Fred Wilson etc. 

Later, the rise of social readers, such as Twitter or Facebook feed, provided a new, relevant set of articles endorsed by my personal and professional contacts. 

Then since late last year, I started using topic-based readers. Prismatic, which algorithmically recommends articles based on my viewing history, has allowed me to discover new sources of interesting articles that my friends, or my existing list of subscribed blogs did not cover. Quora and Quibb, which present an interesting mix of topical and social link-sharing and discussion, also have became an growing part of my diet.  

In an age where we are bombarded with information, Clay Johnson, author of Information Diet, pointed out an interesting statistic at his presentation at SXSW (his slides embedded below). Online media sites that take a more polarized view have been attracting more audience than those that present a balanced stance. Given a main business model on advertising, these sites in pursuit of more viewership, are becoming even more opinionated. 

We as consumers of information, sometimes read to seek affirmation rather than the truth. This, combined with the economic force of driving advertising revenue, is leading to a polarized landscape in media. Hence, Clay believes that, consumption of information can sometimes lead us to be more close-minded rather than more informed. 

I am not sure the solution to this, but this is an alarming thought to me.

One thing I’m seeking to do more, is to read from individual bloggers, balancing my intake of information from large, advertising-driven media businesses. Without the motive of winning advertising dollars, non-advertising driven content can be considered as the “organic food” of our information dietSeek out and share articles or blog posts written by people that are presenting opposing views. Welcome differing views, and challenge the loud, mainstream voices with their assumptions. 

On the same thread, I believe encouraging more people to participate in the dialogue would also help. Blogging is not only a great way to build an audience and share a viewpoint, but more importantly, becoming a creator of content has allowed me to be more conscious of what I consume. Getting into a habit of creating helps also to shift time from consuming to creating. And every small step counts. When it comes to consumption, perhaps less is more. 

It’d also be cool to have a tool that analyzes and reports the weekly information diet, something akin to RescueTime but for browsing history. Just like going on a real diet, the first and most effective step is to know what we eat. 

As Google Reader closes, I want to take this opportunity to rethink my information diet, and strive to become a more conscious consumer of information. 

———————————————————————————

How is your information diet like? Are you aware of what you consume as information? I’m curious to hear about it in the comments! 


Photo Credit: Stadshem 

// High leverage opportunities//

image

Recently I caught up with a former coworker from the investment bank where I spend my first three years of my professional life at.  Speaking with her about decisions at cross-roads, now 5 years into her finance career, it reminded me of a lot on my journey so far. What Chris Dixon recently said really resonated in me: 

Great institutions can prepare you for great things. Credentials can open doors. But don’t let them become an end in themselves.

This was a mindset that had trapped me for longer than I’d like to admit. Little did I know, small steps that started two years ago gradually freed me, as a rewarding career unfolds in front of me.

My Story

As early as my first year in college, I was told to start getting internships to “build up my resume”. I started sending out cold emails to banks, consulting firms, government agencies like my peers, hoping for the best that I get a good internship. I envied over peers’ whose parents’ connections got them into the top investment banks. Even years later when I worked at a top investment bank, I continued to wonder how my next opportunity can make my LinkedIn profile look better. 

Unexpected twists and turns in life, started with my then employer, a VC firm, closing down five months after I started working, provided me with an opportunity to experiment.  Instead of plunging myself into another round of job interview, I decided to take a 6 month contract with a startup in Beijing. While I had enjoyed the fast-paced environment of the finance industry, but had always longed for something more

Fast forward two years, today I’m happily working in a fast-growing startup with an incredible group of people. It is an equally, if not more fast-paced environment than that in finance, yet with so much more. Often I pondered upon how my past decisions led to where I am, I realize my journey in the past two years was defined by seemingly random episodes that turned out to be high-leverage opportunities. 

The mindset of life as an experiment

Instead of the highly sought-after paths, such as going to business school, lesser-known choices seem to reap unexpected reward. Little did I know, my venture to Beijing was a first step. From it, I gained a good taste of how a startup life can be, and became hooked on the idea of starting a software business. Because it was intended to be a 6-month commitment, I felt free to experiment without rushing into a decision to make a career shift. When first breaking out of the “credentials trap”, making small, reversible baby steps helped propel me ahead. 

Think of life as a prototype. We can conduct experiments, make discoveries, and change our perspectives.

- Tim Brown, Change by Design

Believe in the abundance of the world

Shortly after my gig in Beijing, I had coffee with an entrepreneur who later became one of my business partners. Together as a team of three, we built a product from scratch; a hardware and software solution for mom-and-pop shops. I conducted direct sales and sealed partnerships with multinational advertising agencies. Jumping in head first, working without a salary, falling flat on my face, I learned more about the business world in that year than 3 years in finance. 

When seeking out the less-trodden paths, often times I found myself exploring in the dark. Yet, when you stumble and fall, usually the ground is less hard than you think. I’ve found that, when we pursue something whole-heartedly, the world opens up and people are in general more willing to help than I’d expect. Brene Brown, a social researcher specialized in vulnerability and couarge, discussed in her amazing TED talk that, when we become vulnerable, we make strong connections to other people. It is this experience that ingrained in me an "abundance mindset", rather than a scarcity mindset.

Many breakout opportunities are not immediately obvious

After failing at my startup, I took a trip to the Bay Area, where I met David and Christian, the founders of Hackbright Academy. It was two weeks before the first batch was starting, and after meeting them, I could not shake off the thoughts of joining the program. The rational side of me yelled, “Hey! It’s a new program, there is no proven track record, there are not enough information about the founders on Google search results!” As you probably guessed, the irrational side of me won. 

Breakout opportunities are what accelerates your career.

- Reid Hoffman, The Startup of You

Programming has been one of the highest-leverage skill I’ve picked up since college, and thanks to an immersive, intensive three month foundation, my career has taken on a whole new trajectory. We are trained biologically to perceive risk as higher than we think. It is a protective mechanism to tell us to avoid danger at all costs. However, good things can happen when we learn to embrace risk. 

Seeking out high-leverage opportunities in our personal lives 

One of my highest return experiences was a 10-week, 200-hour intensive yoga teacher training course. Having practiced yoga for 6 years, I was able to, through the training, strengthen my practice and deepen my understanding in the philosophies behind the 4,000 year old tradition. It revitalized my body after working 90-hour week in my former life in banking, and provided me with new techniques in handling stress and tension in daily lives. When pursuing a fast-paced career, I continue to find my daily time on the yoga mat a magical, cherished moment of restoration. 

By freeing from the mindset of the credentials trap and seeking out high leverage opportunities, I have embarked upon a rewarding journey. Would you take a baby step today?

Have you ever experienced the ‘credentials trap’, or are you facing a crossroad in life, which you feel pulled by different expectations of the world and your own passion? I’d love to hear from you. 


Photo Credit: Jon Neal

// Key Takeaways from the Marketing and Getting Traction Panel at Startup Product Summit 2013//

Here are the notes for the Marketing and Getting Traction Section at Startup Product Summit 2013.  Omissions and errors are mine (please let me know if you find any, thank you!), credit for the wisdom is entirely the speakers’.   

"Your Product is your Marketing, and You are Your Product" - Eric Kim, Co-Founder & CEO, Twylah

  • Essential foundations of a personal brand:
    • Thought leadership by expressing yourself around themes you want to be recognized for.  
    • Story-telling: Personal brand comes from not just conveying facts but weaving a story.
    • Content-marketing: Convey something of value to your audience. Your content should be educating, inspiring or calling to action.  Identify a segment of audience and project content that’s valuable to that segment.
  • How to cultivate personal brand:
    • Engagement.  Have a conversation around the topics you are interested in.
    • Community.  Allow and encourage interaction and discussion amongst your audience.
    • Custserv: Show the love by being responsive to customer / audience feedback.  Quoted Buffer as a good example of incorporating customer happiness as a core of the product.
  • How to leverage the brand to help or make a business
    • Ask: call to action.  
    • Upsell: selling premium content or related services.
    • Downsell: lower barrier by decreasing the length of commitment.
  • Building a brand is about developing relationship. By consistently laying the foundations, it enables the individual to deliver authority, credibility and familiarity
  • How to get content to right people when first starting out.
    • Content/audience fit.  Publish content and people have something to respond to
    • #hashtag on Twitter is a great way to reach out to audience, outside of the followers count
    • Brainstorm by asking questions. What are the three things that define you, that you’re passionate about, that you can create or curate
  • Trustworthiness springs from familiarity and human touch.  An individual’s brand identity is something that builds up and follows him over the course of his life and career.

"Make Your Numbers Go Up: How to Optimize for Conversion & Retention on Mobile" - Mariya Yao, Founder & Product Strategist, Xanadu Mobile

  • When thinking about optimizing metrics in a product, there is a dichotomy of:
    • Local maxima: am I building the product right?  What should we A/B test?  What platform? How to improve retention?
    • Global maxima: am I building the right product?  Am I in right market?
  • Two strategies in evaluating a global maxima:
    • Benchmarking.  Top down approach that starts with the overall industry landscape.
      • How do mobile users spend their time? What are the fastest growing app categories? What apps have most loyal users? Why do certain apps retain well and how do I use that into my app? What are the most profitable apps segment? Where in the world are smartphones adopted the fastest?
      • What are the trends/opportunities?
      • Where are the danger zones? Why have companies failed there?
      • What can people do on mobile they weren’t able to do before?
      • Why are people loyal to these apps?
    • Behavior. Bottom up approach.
      • Apps are either creating or replacing behavior: A case study on Foursquare and Instagram.  
      • Foursquare was creating a new behavior (check-in) that has no prior parallel, which explains the pivot from check-in in 2009 to explore in 2012. When creating a behavior, focus on delivering value or providing strong incentives such as saved money, time or exclusive/perks.
      • In pre-Instagram era, 76% of people were using their phones to take photos. Mariya examined a detailed flow of taking a photo and sharing on facebook, and contrasted it with how Instagram made the process significantly shorter and easier.
  • Are you solving a problem or a nuisance?  A common pitfall in startups is that they are building a company that is solving a nuisance instead of a problem.  When solving a nuisance, in order for such product to succeed, the product needs to be better than the existing solution by magnitudes (eg, everyone complains about Craigslist but in the end tolerates it). A good way to distinguish between problem and nuisance is: Have they [the users] paid for a solution before, or spent a lot of time finding a solution or making a solution themselves]?
  • Mariya’s presentation is available here.

"Reach Escape Velocity through Lean Content Marketing" - Guillaume Decugis, Co-Founder & CEO, Scoop.it

  • ”Content marketing is practice of creating content relevant to your brand to gain greater visibility in search results and in social channels” - JD Lasica, Social Meida Biz
  • 4 strategies that lean content has worked for Scoop.it
    • Leverage SlideShare’s natural distribution to share your vision: Despite having a relatively small followers count, Scoop.it’s content on SlideShare has gathered significant views.  
    • Guest post to get distribution for your idea. Identify blogs in your niche and segment which are interesting.
    • Answer Quora questions that related to your field.  Answering on Quora is like blogging for a known, existing and savvy audience, who already have questions.
    • Content curation.  Starting point, leverage what you already do (read), express your expertise & develop it, helps you identify original topics for content creation.  If you don’t know what content to create, start with curation.

"Monetization: How Buffer went from Idea to Revenue in 7 weeks & 50K users in 8 months"

  • Leo shared three key stories/lessons from building Buffer so far.
    • Validate first, code later.  The story of Joel, cofounder of Buffer, validated his idea of Buffer before writing a line of backend code, by putting up a landing page and testing clicks on sign up and pricing.  See more here.   
    • Working with percentages.  When doing business development and getting press, Leo suggests anchoring expectations with percentages (out of 10 emails, expect ~20-40% response rate), to avoid frustration and improve resilience in mindset.  He talked about the mindset helped him write 350 guest posts in the first 9 months of running Buffer, and getting press every 3 weeks.  
    • Experiment with pricing.  He encourages the audience to test frequently pricing plans and points what works for the product, while always be great to existing users.  

"Launching and Getting Users" - Jameson Detweiler, Co-Founder & CEO, LaunchRock

  • Jameson told his journey of first 42 days of running LaunchRock, from StartupWeekend to launching on Day 5, getting on TechCrunch on Day 7 and campaign at SXSW on Day 42.
  • How to effectively launch and get users:
    • Be sexy [ beautifully designed product ], flirtatious [ LaunchRock’s traction is in part thanks to the wait between users’ signing up and the product is ready ], exotic [ build something new and unique ].
    • Be targeted and specific in the value proposition; that makes it easy for people to talk about your brand.
    • Be nice to press: do the related research, and make it easy for the press to write about you.
    • Launch and listen to customer feedback.
    • Be authentic when telling your story. 

Notes on other panels:

// Key Takeaways from the Roadmapping and Execution Panel at Startup Product Summit 2013//

Here are the notes for the Roadmapping and Execution Lightning Talks and Panel at Startup Product Summit 2013.  Omissions and errors are mine (please let me know if you find any, thank you!), credit for the wisdom is entirely the speakers’.   

"Building a Great Product Through Communication" - Joe Stump, Co-Founder, Sprint.ly

  • Product manager’s role is to capture, communicate and distill product ideas, and mediate between business stakeholders and makers. 
  • When building a product, pick two out of the three: quickly, correctly, cheaply.  Joe later mentioned on Twitter that he would pick quickly and correctly, as paying for quality is no brainer. 
  • "Want to increase innovation? Lower the cost of failure" - Joi Ito 
  • Empower every developer to commit things to the product through non-blocking development (NBD).
  • Advocate the move to 100% asynchronous communication because current approach is broken (needs human input to track reality) and remote teams are becoming more common.

"Raw Agile: Eating Your Own Dog Food" - Nick Muldoon, Agile Program Manager, Twitter

  • Twitter does dog-fooding by allowing developers deploy to internal server. Dog-fooding allows:
    • gathering real data from real (though internal) users. 
    • increases incentive to produce quality shipped code.
    • better feedback.  He found that feedback in dog-fooding environment is generally more constructive. 
    • keeps momentum through a positive reinforcing loop of continuous deployment and feedback. The team gets 50-100 feedback from internal users each day. 
  • How to decipher and sift through the volume of feedback.  Look at only the “love” feedback, then all the “hate”, then discard the middle, categorize and show to the whole team. 
  • Other important aspects in dog-fooding: 
    1. Automation. Allow deploy more frequently especially internally.  ”On any commit, deploy internally.” Avoid accumulating technical debt.
    2. Visibility. Record progress and share on a wiki. 
    3. Speed. Minimize cycle time (from to do to in progress, to done). 

Best Practices for Architecting Your App to Ship Fast and Scale Rapidly” - Solomon Hykes, Founder & CEO, dotCloud

  • 3 things to aim for in architecting your app: speed (continuous deployment), scale, future-proofing (be prepared for things moving very fast, avoid bottleneck and need to refactor when adding every new feature).
  • What are the patterns/strategies in getting to these three goals?
    1. Be aware of trade-offs. There is no silver bullet; always trade-offs and prioritization.
    2. Trade-offs evolve over time.  Priorities change. Be aware of assumptions and revisit them from time to time.
    3. Trade-offs differ from team to team.  Be aware of bias in different teams. Always keep ownership of key decisions. 
  • Put yourself in a position where you are embarrassed, and things are going to happen faster.  

"Rocket Powered Bicycles: Avoiding Over and Under Engineering your Product" - Chris Burnor, Co-Founder & CTO, GroupTie & Curator, StartupDigest

  • A product connects business priorities with user experience. 
  • Proposes that instead of Minimum Viable Product (MVP), think about Product: Viable Minimum (PVM).  Focus on viability. 
  • A scientific method to approaching product roadmapping.
    • Idea: think about business priorities, user experience.  Do not let technical decisions drive your product.  Let product drive your technical decisions.  
    • Test: Viability of the solution is whether it solves the problem it’s setting out to solve.  Determine what level of viability is suitable in different stages: GroupTie’s first viable minimum was a keynote presentation that was sent to potential customers.  
      Scale of tests will vary.  Lack of big tests means the lack of breakout growth/ideas, lack of small tests means the team is doing too much. 
    • Conclusion:  Debriefing phase is vital, share test results with the team and learn what it means to the idea. Testing without debriefing is like “talking without listening” in a conversation. 
  • An unusual example of a PVM is Apple.  Product first: cares about user experience and business priorities.  Viability second: it just works.  Minimalism third: wait till a technology is ripe before adding to the product (no LTE for a long time, no RFID).  

Notes on other panels:

// Key Takeaways from the Design Thinking and Rapid Prototyping Panel at Startup Product Summit 2013//

I had the privilege to attend the Startup Product Summit yesterday, it was a great lineup of speakers and a full room of buzzing energy and great conversation.  Without further adieu, I’d like to share some key learnings of each panel.  

Please let me know if I omitted or made any errors in the references. Credit for the good stuff is entirely the speakers’ (link to twitter handles are included on each name). 

Turning Mediocre Products into Awesome Products” - Jonathan Smiley, Partner & Design Lead, ZURB

  • Ideation and iteration can ”turn mediocre products into awesome products”.
  • Discussed a full spectrum of research from market-driven (focus group, survey) to user-driven (remote teaching, usability teaching)
  • Importance of sketching, a lot, aim for speed and volume, then critique
  • Advice to the audience: ”do 10 more sketches ( more ideation is always better ), build 1 more prototype, get 1 more round of feedback, ask 5 more customers”

"Being a UX Team of One" - Vince Baskerville, Product, Lithium & Co-Founder, TripLingo

1. Internal politics is a common challenge as a UX team/professional.  Learn to manage expectation of different internal stakeholders and keep everyone in the loop. 

2. Don’t listen to what customers are saying.  Users’ claims are often unreliable.  See what they are doing.  Understand the underlying issue.  

"Validate Your MVP on Paper" - Poornima Vijayashanker, Founder & CEO, BizeeBee & Femgineer

- 2 Reasons MVP Fail
1. 
Fail to figure out how to provide a simple value proposition that differentiates your product from your competition
2. 
Fail to figure out who their early adopter are.

- Early adopters are people who aren’t using the competitor’s product. Don’t want to take time to switch over.  

- Steps on usability testing 

  1. Explain the problem. What you are testing. How they are helping. Get them excited about the idea. 
  2. Set expectations. Make them comfortable.
  3. Communicate intention (what exactly are you testing and specific feedback you are looking for).
  4. Thank them for their time. Follow up regularly.    

Poornima’s slides are available here

"Everyone’s Customers Are Wrong" - Evan Hamilton, Head of Community, UserVoice

  • Data doesn’t tell the whole story.  Analytics are bandaids because we can’t watch our customers.  
  • People don’t tell the whole story.  Identify who the users are, where the feedback are from.  Are they: paying/freeloaders? Using product in the intended way? Using main features? Early adopters / ‘tech fanatics’ (who are not likely to stay on a product for the long haul)?
  • Combine data and customer stories.  Customer feedback / feature suggestion usually leads from a deeper issue.  Find out what the actual problem is by understanding the underlying need. 
  • Don’t lose track of your creative mind by getting lost in data rat-hole.  Don’t chase 1% when you can get 15%.  Not just A/B, but try something crazy.  Try big bold things along with incremental fine-tunes. 

"Designing for Everyone: The Craft of Picking or Killing a Concept" - Miki Setlur, Product Designer, Evernote

  • Everyone use product in many different ways.  A useful strategy is to segment users into business, partners (e.g., app stores for Evernote’s case) and users. 
  • Figure out what each segment cares the most about: Business / Partners - acquisition, retention, engagement, revenue.  Users - being faster, better, happier. 
  • Case study on how Evenote’s design process stroke balance between business goals (monetizing) while being sensitive to user experience and goals (finding things faster). 

Other relevant points 

How to access willingness to pay during pre-product interviews.  Get the first dollar within the trial period.  Provide clear value proposition from the get-go.  

How to get good feedback.  Be specific in what feedback are you looking for.  Instead of asking in general ‘what do you think of the prototype’, ask whether they are confused on what stage, what was confusing. 

Tips on prototyping. Put more emphasis on story telling than illustrating. 
For remote testing, use keynote as prototyping tool, screencast the keynote.

On the tension between product vs. business goals in roadmapping a product.  Early stage products make sense to focus on product.  Once reached product market fit, it makes sense to lead with business goals such as, acquiring, converting, retention customers.
Also mentioned was a tool called Impact Mapping

Read on:

aquinomarlon:

#powerofyou #positivity #life #quotes #instagood #instaquote #struggle becomes #Reward

Something better

aquinomarlon:

#powerofyou #positivity #life #quotes #instagood #instaquote #struggle becomes #Reward

Something better

(via couragehopestrength)

blog comments powered by Disqus
I work at Buffer on metrics/growth. Here I share about startups, personal improvement and wellness.
I'd love to hear from you!