There’s no two ways about it: scaling your API from an MVP product to one that can attract (and support) thousands of customers of all sizes is challenging, and the work gets more complex as the company grows.
At a recent Nylas Developer Event we explored this idea of building systems that scale with Liz Acosta, Software Developer at Twilio; Mike Atkins, Software Engineer at LaunchDarkly; Keith Horwood, CEO and Founder of StdLib; and our own CTO and Co-Founder Christine Spang.
Moderator and developer community evangelist Mary Thengvall asks the speakers to address pressing topics like:
Grab a bag of popcorn, pour your favorite beverage, and sit back to enjoy the fun repartee of the panelists.
Keith: The thing that ‘scale’ most evokes in my mind is an existential fear, mostly because most of the things associated with scale are words I can barely pronounce, like, Kubernetes. There’s also all these words like ELB, EC2… a lot of stuff that, especially in my earlier tech days, made me constantly feel like I was way out of my depth. On the other side, I also look at ‘scale’ with optimism. Whenever scale becomes a problem in the company, it’s a good problem to have. Obviously, you’ve hit some level of success, so from a business perspective, that’s cool. It’s a mix of fear and optimism at the same time.
Liz: I think my initial response is ‘I get a headache.’ Besides that, there’s optimism — ‘Cool, we’re growing!’ I think it’s also really exciting because it presents brand new challenges — something new to work on that requires stretching your imagination. That’s where the creative part comes into development.
Mike: One thing that I definitely feel is the sense of dread and foreboding when I get the pager and am on-call during times that we’re scaling, but it’s also exciting to see the ideas in your product get more fleshed out, as well as follow along with the additional use cases customers come up with. Seeing those new developments is always really exciting.
Spang: I’ll definitely second the vibe of excitement, but also, ’scale’ makes me think of things falling over all of the time. This is exciting but also, sometimes, you need a break from the excitement because you just want everything to work now.
Keith: Honestly, I would tell them to go for a run. If there’s a tough decision that’s coming, or if there’s a lot of stuff on your mind and you’re freaking out, some people turn to meditation, and I actually find running meditative. Find something that can get your blood pressure down.
Spang: Ask for help. One of the really amazing things about Silicon Valley is that there is a community of people here that really want to help each other. Scaling is one of those things that people want to help you with, because your product is taking off! Just feel it out — find people you can trust and tell them about your problems, because they can often help you solve them.
Liz: Don’t try to pre-optimize for it. Start writing that first draft — just get it out and then you can edit from there. If doesn’t have to be perfect right away — you can always iterate on an idea. Don’t try and optimize before anything even builds, because you are just going to end up frustrated.
Mike: Be aware of things that could be a problem in the future and make sure there’s a share awareness of these potential issues. The worst thing that can happen is somebody else is on call and there’s a scaling issue that you knew about, but then they have to figure it out in a panic. If it’s documented, there’s a chance that they’ll recognize it when they see it and be able to fix it more quickly.
Mike: I think it’s probably the servers and events protocol. Instead of having a super read heavy workflow where clients are constantly calling us, you can just send them data on the changes and then they never talk to us. It’s great.
Spang: It’s hard to pick just one. We use the same programming language that we used on day one, which is Python. It’s been great and we haven’t had to re-write anything — it’s been able to evolve as we scale. We’re also really heavy users of MySQL. We picked it specifically because we knew it had been used at scale and we didn’t want to solve those problems in the early days of the company like ‘How do I keep this many bits in the thing?’
Liz: For us it’s Datadog. It allows you to keep track of those precious metrics so you don’t have to write all of the internal infrastructure for that, it just comes out of the box.
Keith: Our company wouldn’t exist without AWS Lambda. Especially the first go-to-market version of our company.
Liz: As developers, we sometimes forget that there’s other people who do other jobs in our companies! It’s really important to maintain a dialogue between developers and maybe what marketing or sales is doing. Sometimes, sales has gotten in touch with a new customer and they could say, ‘[This new customer] is going to bring in this much traffic, and this much data so, hey, developers, can you help us with that?’ I think, remembering that your company is an entire ecosystem made up of different organisms and we’re all doing a different part means that you can keep that dialogue open and keep an eye on what the company is doing so that as a developer, you’re not just reacting but you’re working in collaboration with the rest of your team.
Liz: Having a really strong mission statement that you develop in the beginning is key here. Then when you have a change coming up, you can evaluate, ‘Is this going to impact these customers, and if so, how?’ and return to that mission statement. At Twilio, we say ‘We’re not just selling like a product. We’re selling trust.’ From our smallest customers to our largest customers, we know that if we’re going to change something, we have to make sure that that’s still aligned with our mission and building trust with our customers.
Trust can be anything from making sure that your smallest customers are not impacted by a change you make to accommodate larger customers. It’s also about letting customers know when something is going to change and being transparent about it. Again, when you have a really strong mission statement then, when you’re asking questions like, ‘What should we do next?’ and ‘How do we do this?’, you can keep returning to that mission statement and ask, ‘Is this decision in line with that?’
Mike: I think as you gain more customers you have to take more care with the message you put out. As more and more people are looking at the things you’re publishing and the code you’re writing, you have to be more and more careful with the quality of what you’re releasing.
Spang: I think there’s also a couple of things that are unique to API and SaaS businesses. One is super basic: If people are building things against the platform that you’re providing, they hate it when you change things and don’t tell them — it breaks their stuff! This is rule number one — You have to tell your customers with a lot of advance notice if you’re going to change something that might break what they’ve built. You should be able to go into the data that you already collect and get a good idea as to who might be affected by a change that you’re making. Then, proactively reach out to them to make sure they know what’s changing or what they need to do to make sure that their code keeps working reliably and that they feel supported. People are relying on you — you owe it to them to make sure that they are able to adapt with your platform.
Spang: One key piece is that your engineering team is not the only part of your company that’s going to need to scale as you grow. You need to think about what is the right ratio between engineering and all the other departments that are needed to support your product. For example, we have a developer success team and as we get more customers, we not only need more engineers we also need more customer success engineers.
Mike: Shifting to written communication as the company gets larger is pretty effective. Having a bunch of verbal, in person meetings, works really well when it’s like 5 or so people, but once you start growing, not everyone could be in the same room at the same time, all the time. Investing the time and writing down what you’re planning to do and when you’re planning to do it often goes a really long way towards everyone else having a shared awareness of where the company is going and what you should be working on. Thoughtful, well-written documents — not just Slack channels — are particularly helpful.
Spang: Write meeting notes when you have a meeting and document any decisions that were made in that meeting. Designate someone as a facilitator for every meeting to keep things on track and then have a separate note taker. In the early days we were five people in a room and we never wrote anything down. If something came up, we just talked to each other and worked it out, but eventually, not everyone could be there, but everyone needed to know what was going on.
Liz: Having really clear and thorough documentation is really helpful. Documenting decisions gives new employees insight into the company history as well — all these little ‘magic tricks’ that other people know but you, as a new employee, might not be familiar with.
Keith: The leaders that I’ve respected the most and the communication styles that seem to be the most effective are generally ones where the managers are almost over-communicative. They’re constantly communicating the mission & vision, and making sure that their team is on track every day. I always found that the teams that I was on, especially those that were full of really talented, enthusiastic people, tended to go a little sideways was when you don’t have that top-down reiteration of, ‘This is what we’re doing. This is why we’re doing it. This is why you need to pay attention.’
Spang: I think it all comes down to culture. In order to create and maintain a great culture, you have to articulate your organizational values — write it down and tell every single person that you’re interviewing as well as reinforce it with your current employees — because at the end of the day, the experience that customers have is going to be determined by the other people in your team and what they choose to do day-to-day. Just like you can’t be in the room for every single conversation, and decisions that’s being made, you can’t be in a room for every other interaction that’s happening in your company. You have to invest real effort in scaling how your team should think about what they’re doing, and what they should be spending their time on.
Mike: It’s important to acknowledge that as a company grows, it’s going to change. As the company grows, trying to make it exactly the same as it was when it was 5 people isn’t realistic. Similarly, as a company improves you have a bunch of more smart people that you’re working with and allowing them to have an influence on what the company is. I think it’s also super important.
Building a scheduling application that displays the organizer’s available timeslots seems like a simple problem…
Explore the transformative impact of AI, the evolution of global connectivity, and the reshaping of workplace culture in the digital era in Nylas’ 2024 predictions.
Use the Nylas Email API and Ruby to group email threads into a single view, and easily access complete conversations within your app.
Email API
Calendar API
Scheduler
Contacts API
ExtractAI
Salesforce email integration
Google email & calendar integration
Outlook email & calendar integration
IMAP integration
Calendar management
Embedded contextual email
Scheduling automation
Automated outreach
Intelligent data extraction
Ecommerce
Telehealth
Real estate
Financial technology
Educational technology
Expense management
Customer relationship management
Applicant tracking system
Communications platform
Marketplaces
Document management
Docs
SDKs
Tutorials
API status
Changelog
Support
Blog
Events
Guides
Inspiration gallery
Partners
Build vs. buy
Customer stories
Security & Compliance
Why Nylas
Newsroom