Jory MacKay on September 28, 2021 Everything Product Managers Need to Know about Scalability: 10 Ways to Design and Build for the Future
Scalability is a big deal in enterprise development. When you’re working with massive companies, get a significant spike in users, or land a great white whale of a customer, you want to know that your infrastructure, tech, and team can handle the increased load.
Unfortunately, smaller or younger teams with limited resources rarely think about this magnitude of scale. Instead of building a foundation for more users, data, and load, they get caught up in the more immediate issues of running a company.
Jump to a section:
Yet, along with keeping your current customers happy, first impressions matter. Research has found that anywhere from 60–88% of users won’t give an app a second chance if they have a bad first experience.
Luckily, there’s a middle path between growing today and planning for a future you might never see. With the right tools and processes, it’s possible to make scalability a part of your DNA–no matter your current size, customer base, or market.
This post isn’t a deep dive into the technical specifics of scalability (we’ll leave that to more capable DevOps engineers to write about!) Instead, it’s a primer to help project managers understand scalability and how to master the balancing act of building new features and building robust processes that enable endless growth.
What is scalability? A primer on how to handle company growth spurts
Let’s start with a simple definition to get us all on the same page: Scalability is your ability to efficiently scale resources up or down to meet demand.
Everyone can throw money at a problem until they’re broke. Scalability is about doing more with less. With the right tools and processes in place, you don’t have to worry that growth, virality, or variable load will tank your product (or your company!)
Scalability is about doing more, with less.
Scalability is essential in everything from your business to your technical stack and even your team. Here are a few examples:
As a business, you can think of scalability as the ability to serve more customers. If you sell hamburgers and fries, can you make enough hamburgers and fries if your customer base increases by 5X overnight? What about 10X? 100X?
As a technical team, scalability is a slightly different beast, but the idea is the same. Can your app/tool/site handle a spike in traffic without crashing or slowing down to a crawl? What will happen if concurrent queries to your database jump from 1,000 to 10,000 in a matter of seconds?
Finally, as a project manager, team lead, or CEO, do you have the network, resources, and processes to grow your team and handle new business or users? (If you’re unsure, we wrote an entire post on scaling product teams here!)
These might sound like serious risks. But the truth is that these are usually good problems to have.
Struggling with scalability means you’ve done something right, and more people want what you’re serving (whether burgers or online services).
As The Code and Bootstrapping Podcast says:
“Scalability problems are the first world problems of first world problems. They only happen when everything is going really well, and you've already done something that almost all companies fail to, which is make something that so many people want that they're overwhelming your capacity to supply it to them.”
However, just because scalability is a ‘good’ problem doesn’t mean you can ignore it or ‘cross that bridge when you get there.’
The ‘Two P’s’ of scalability: Performance and processes
Scalability is most often seen as a technical problem. However, just as important as having the right tech stack to handle growth is having the right processes in place for your team.
That’s why we like to break down scalability into the two P’s: Performance and process. Let’s dive into both.
Performance: How your engineering team thinks about scalability
As a software company, your technical team thinks about growth in a completely different way than you do. As the team at Built In write:
“When a CEO learns his company will appear on the television show Shark Tank, the natural reaction is excitement for potential hockey stick growth. A CTO’s reaction? ‘Oh no.’”
If usage spikes suddenly, will your service slow to crawl? Or simply give up and crash under the load? If some viral news story sends you 30X your expected traffic, can you efficiently handle those new readers? Or will you be burning money just trying to keep your site online?
Rather than the excitement of new users and more revenue, growth means technical uncertainty.
When there are tens or hundreds of factors influencing the speed and effectiveness of your software, any one of them can tip you over the edge. How you handle all those situations is how you build scalability into your software.
There are tons of posts and debates on which tech tools and system infrastructure is best for scalability. As a project manager or non-technical leader, you don’t need to know the specifics of each. However, you should understand the basics of how your technical team handles issues of scalability.
Horizontal vs. vertical scaling
Software scalability means adding more computing power to your infrastructure. However, there are two different ways your technical team will talk about doing this: horizontal or vertical scaling.
Horizontal scaling is when you add more machines (or servers) to your resource pool (also known as ‘scaling out’). Vertical scaling is when you add more power (CPU, RAM) to your existing machines (also known as ‘scaling up’).
The main difference between the two is that horizontal scaling requires breaking a piece of code into smaller pieces that can be run in parallel across multiple machines. This can add redundancy and flexibility to your system but also complexity.
On the other hand, vertical scaling keeps your code together but just runs it through a higher-spec server. This is simpler and cheaper but means you’re limited to the power of a single machine. If you need to upgrade or something goes wrong, you might be in trouble.
Processes: How you need to think about scalability
Your technical team is the best resource for handling the performance aspect of scalability. So if you’re not actively involved in that process, how can you help?
Solving scalability is all about removing uncertainty. But instead of doing so through tools or updating your stack, you can focus on the human element of scalability: Processes.
As you build a more extensive and more complex system, what does your team need to know?
- How are they preparing for handling more data?
- Where can they learn about the system architecture and how to flag issues?
- Do you have playbooks for when mistakes happen or load increases?
- What can you automate to help give your team more time for critical issues?
With Planio, you can create custom knowledge bases and wikis to document all your team’s processes. Think of it as an ‘idea gathering place’–a centralized, customizable location to store information.
For example, you could use a Planio wiki as a runbook for when scalability issues come up. This means that whenever common problems arise–whether it’s a spike in traffic or a massive increase in API calls–your team knows where to find the process for handling it.
What’s especially great about using a tool like Planio is that because it’s also where you track issues and run projects, you can directly link your processes and runkbooks to specific tasks, projects, code repos, or shared files. This gives everyone the context they need to tackle problems properly.
10 ways product managers can help support (and encourage) scalability efforts
Without downplaying the technical aspects of building software that works for millions of people, performance problems can often be easier to tackle and solve than process ones.
There’s usually an explicit action to take when your system needs more juice. But without scalable processes and shared resources, both your team and your app can quickly get overwhelmed.
Scalability issues are often hiring, planning, resource management, and process issues in disguise.
Here are ten ways to address those issues and make scalability a part of your team’s DNA.
‘Unscalable’ acts build your culture. Scalable ones build your business.
1. Know when to think about scalability
Scalability isn’t always an issue that teams need to care about. At least right away.
If you haven’t found product-market fit, it’s probably more important to experiment with different features or pitches before planning for your app to go viral.
As Y Combinator founder Paul Graham writes, startups need to do things that don’t scale to get noticed.
For example, talking to every customer, obsessing over hiring the right team, and sometimes even manually doing what you want your software to do automatically later:
“When you only have a small number of users, you can sometimes get away with doing by hand things that you plan to automate later. This lets you launch faster, and when you do finally automate yourself out of the loop, you'll know exactly what to build because you'll have muscle memory from doing it yourself.”
In the early days of Stripe, the only way they were able to offer ‘instant’ merchant accounts to their initial users was by the founders manually signing them up for traditional merchant accounts in the background.
Yet while these sorts of manual actions can be great for product experimentation, by the time you’re working with a mature product, they’re downright dangerous.
As a product manager, you should be on the lookout for warning signs that it’s time to think about scalability. Ask questions like:
- Have you taken on too much technical debt in the name of growth?
- Are simple updates being roadblocked by complex code?
- Are you experiencing lag or downtime even with your current usage?
Set up systems to monitor and flag scalability issues so you won’t be adding resources too early or too late.
2. Balance ‘building’ sprints with ‘scalability’ sprints
Knowing you need to start addressing scalability is only half the battle. The rest is working it into your current backlog, roadmap, or code reviews, which isn’t always easy.
Scalability isn’t sexy. There’s no external recognition when you do it well, but lots when you do it poorly. So instead, it’s easier to prioritize new features, development speed, and growth.
However, scale exaggerates issues. What felt like a minor annoyance when you were serving tens of customers can take your entire product down when you jump to hundreds or thousands.
That’s why it’s a good rule of thumb to address (or at least consider) scalability from the beginning. This doesn’t mean delaying a product launch to ‘perfect’ your infrastructure. But instead that each sprint should include the question: Does this scale?
If it doesn’t, and you don’t have a simple solution, document what you’re doing and why directly on the issue. Then, create a custom ‘scalability’ tracker in Planio–this is a simple way to categorize different issue types.
Later, when you schedule a ‘scalability’ sprint, you have a backlog of issues to work through.
Solving scalability is all about removing uncertainty.
3. Focus on scalable processes (and leave the performance issues to your team)
Your technical team doesn’t want you jumping in and suggesting ways to optimize performance. Leave that area to their expertise and instead focus on the processes that are necessary as you scale.
This means:
- Sprint planning and retrospectives: How will you keep your sprint planning process efficient as your product and team becomes more complex?
- Feature prioritization: How will you balance prioritizing new features and tackling technical debt?
- Hiring and resource management: Scalability refers to your team as well. Do you have a clear process for hiring or procuring extra resources as it becomes necessary?
- Setting OKRs and other team goals: Complex systems can obscure your product strategy and vision. Do you have a process in place for better goal-setting that keeps everyone focused on the right work?
- Cross-team collaboration and communication: As your product scales, you’ll end up with more teams, more managers, and more collaboration. Do you have a communication plan in place for cross-team collaboration? What about a communication runbook for handling outside requests?
4. Give your team autonomy to learn and prioritize on their own
As your product scales, you’ll inevitably attract a more diverse group of users. Understanding their needs is how you’ll maintain your rate of growth. But if your team still needs to be given detailed requirements, they’re going to get overwhelmed with work.
User interviews are a powerful tool for understanding how people use your product and should be conducted (or at least experienced) by as many people as possible on your team. As we wrote in our Guide to Conducting Better User Interviews:
“User interviews are a formal process for engaging with your users to understand their problems, habits, skills, and desires. More than just a casual conversation, user interviews have specific goals, are recorded for research and future use, and produce actionable insights for development teams.”
Give your team autonomy to conduct research and understand the user better. Or, at a minimum, give them access to recorded user interviews to help them prioritize future sprints.
5. Allow things that don’t scale first (but think of how they will later)
It can be hard to purposefully build features you know will crack under pressure. However, closing off your team from building exciting features because they don’t immediately scale is a bad idea.
Sometimes the ‘things that don’t scale’ are manual processes. Other times, they create technical debt–the ‘cost’ of choosing an easy or limited option now when a better choice would take longer.
Technical debt is a necessary evil when validating ideas or testing out features and shouldn’t be avoided. However, just like taking out a massive loan from the bank, you need to know how you’re going to pay it back.
Ensure you have systems in place to monitor scalability and keep a record of technical debt or ‘things that don’t scale’ in your project management tool and revisit them when you’re sprint planning. You don’t want to start building on top of a cracked foundation.
The question isn’t whether you need to worry about scalability. It’s when to make it a first-class issue.
6. Bring together the right team
Scalability is often a hiring issue. The right team for a project means people with experience building at scale or who understand the trade-offs of taking on technical debt.
As CTO Bernard Kowalski writes on Built In:
“It is difficult to build scalable systems without experienced engineers tuning both parts of the engine.”
As a product manager, this means thinking about your core team at different stages of your product or company. As your product team scales, do you have the right people who will steer it towards scalability?
7. Understand the trade-offs when building a scalable system
In the hierarchy of team concerns, scalability sits below the speed of development and ease of hiring. In other words, you shouldn’t worry about scaling if you don’t have processes in place to build software or hire good people quickly. You need to consider this same hierarchy when your team is talking about scalability.
There’s always a trade-off when it comes to tackling scalability. If you’re a SaaS company and you’re sacrificing development speed to mitigate the risks of a potential massive usage spurt, you’re probably making a bad call.
However, if you’re building a free-to-play game where uptime is more important than new features, that might be a tradeoff you’re willing to make.
Pick a path and stick with it.
8. Be the source of truth for vision and product strategy
Scaling can often add complexity and confusion. But to do their best work, your team needs to be reminded of the bigger company vision they’re working towards.
This means baking the essential elements of your product strategy into OKRs and sprint planning sessions. As a refresher, product strategy includes five components:
- Vision: What’s the long-term goal of your company?
- Challenge: What business goal will help you reach your vision?
- Target conditions: How will you know you’ve achieved the challenge? What metrics can you start exploring today?
- Current state: Where are you right now in regards to your target condition?
- Threshold of knowledge: What do you know now that will help you about the market, ideal user, etc...
9. Look for ways to automate aspects of the customer journey to free up space for ‘critical’ tasks
Scalability is (usually) firmly in the ‘important but not urgent’ section of your priority matrix. Unfortunately, that means there are often tons of ‘more urgent’ tasks that get in the way. To give your team the space needed to focus on scalability, look for ways to automate the ‘urgent’ parts of your product or the customer journey.
Some common examples could be account creation, user messaging, onboarding, bug reporting, or anywhere users might ‘urgently’ need technical help.
10. Keep information about architecture, process, and scaling public and up-to-date
A knowledge base of your scalability processes is only good if it stays up-to-date and is easily accessible. The same goes for any other knowledge management efforts.
Keeping your processes in a Planio wiki means it’s always accessible and that team members can ask questions or comment on specific procedures. However, you also need to incentivize your team to keep these processes up-to-date.
Here are a few tips to help get your team on board with documenting processes:
- Create a culture of knowledge sharing: Build a sense of psychological safety where no one is afraid to ask questions or propose ideas. This is sometimes as easy as changing the language allowed in meetings to be more inclusive, or giving everyone time and opportunity to contribute.
- Incentivize quality over quantity: Show what ‘good content’ looks like and provide resources to help everyone write better.
- Use the right tools: Use a simple knowledge management system (like Planio wikis) that makes it easier for everyone to contribute and find the information they’re looking for.
Sharing knowledge has compound returns on your entire company. The more you can make it a part of your culture, the faster you’ll be able to build.
‘Unscalable’ acts build your culture. Scalable ones build your business.
Scalability can quickly become a divisive topic. Technical teams will say it’s important to consider from day one. While marketing, sales, and leadership will push back until it becomes a real issue.
Yet the truth is that both groups are right.
The question isn’t do you need to worry about scalability, but rather when to make it a first-class issue?
It’s OK to embrace some messy, scrappy growth ‘hacks’ in the early days. Those are what create your company culture and mindset. However, knowing when you need to turn those ‘hacks’ into formal procedures with the tools and processes to back them up is a skill that every product manager should have.
