Last Updated on August 9, 2024
Published on: November 7, 2023.
Amidst the digital era’s inexorable march, data science emerges as a behemoth, its tendrils extending into every industry, from retail to autonomous vehicles. At its core lies the art and science of sifting through data avalanches to unearth insights that redefine decision-making and innovation. Abhishek Chaurasiya is a luminary in this domain, steering the leviathan of data to mold the products that become integral to our everyday lives.
A narrative of inspiration and ambition, Abhishek’s odyssey in data science began at the University of Cincinnati with a degree in Business Analytics. It’s an odyssey that spans a decade, a journey marked by his transformative touch at e-commerce giants such as eBay and Amazon, and most recently, at the delivery service innovator, DoorDash. Here, Abhishek has played a pivotal role, delving into the psyche of the consumer and meticulously crafting market offerings that resonate with unspoken desires.
Distinctive is his entrepreneurial flair—Abhishek co-founded an ed-tech startup, lighting the path for those yearning to carve out a niche in the data science universe. He dons the hat of mentor and judge, investing in the fertile minds that will sustain the future of technology. In the crucible of product testing, his touch is that of an artist, ensuring that each product’s silhouette is a harmony of consumer expectation and strategic vision.
In our exclusive interview, we explore the mind of this data science maestro, traversing the nuances of product testing and the symphony of algorithms that predict success. Abhishek opens up about the challenge of translating academic theory into the pragmatic world of data science and shares his vision for educating the next wave of innovators aiming for the zenith of tech careers.
Join us as we unfurl the tapestry of Abhishek Chaurasiya’s professional saga, a journey emblematic of a data science virtuoso who not only charts a course through the labyrinth of data but also shapes the contours of our interaction with the ever-evolving world of technology.
Hello, Abhishek! Let’s begin. You’ve been with eBay, Amazon, and now DoorDash. Could you walk us through how data science is woven into the product development process at these places? What kind of unique challenges pop up?
In companies like those I’ve previously been part of, data science is intricately integrated into product development. Data scientists are central to many aspects of this process. We rely on data-driven insights to spot market trends, understand customer preferences, and pinpoint opportunities to enhance our products. This intelligence drives the prioritization of product features and shapes the creation of personalized user experiences, such as custom recommendations and content tailoring.
However, the marriage of data science and product development is not without its challenges. The sheer volume and complexity of data that e-commerce platforms—and indeed any major platform—accumulate daily make the tasks of collecting, storing, and processing data daunting. Maintaining data accuracy, ensuring privacy, and safeguarding security are all vital, yet challenging. Data scientists must also strive to build predictive models that remain relevant amidst fast-evolving customer behaviors and market conditions. Additionally, the necessity for real-time decision-making in scenarios like dynamic pricing and inventory management introduces significant complexity to our work.
I’m sure you’ve seen products evolve dramatically thanks to data science insights. Could you give us a behind-the-scenes look at one such transformation?
Certainly! An illustrative example comes to mind from my tenure at DoorDash. My focus was on the Convenience and Grocery delivery sector, wherein customers order retail and grocery products for delivery. Our role was to coordinate with dashers who would go to the stores, collect the items, and deliver them.
During an experiment concerning delivery times, we analyzed the geospatial data of the dashers. Surprisingly, we discovered that the bulk of a dasher’s time was spent not on the road but inside the store itself.
Armed with this insight, we examined the dashers’ feedback on their orders and uncovered that locating items and checking out could take up to 50–60 minutes in many instances. Recognizing this inefficiency, we engaged with our Product and Operations teams to propose solutions: either designate separate dashers for in-store shopping or collaborate with retailers to pre-pick and process the items.
Influenced by the feedback from our leadership, we opted for the latter solution. This pivotal decision has allowed us to reduce delivery times to less than 30 minutes from stores where this system was implemented, marking a substantial improvement for both customers and dashers alike.
Product testing must be quite the balancing act. How do you leverage data science to not just tick the functional boxes but actually hit the mark with users’ expectations?
Data scientists deploy a suite of strategies to ensure that products are not just functional but also resonate with user expectations. Initially, we collect and analyze user data to unearth insights into their behaviors and preferences. This information becomes the cornerstone for designing features and prioritizing product enhancements. A/B testing is a critical tool in this phase, enabling us to measure the impact of modifications and refine our products based on user feedback.
Machine learning models play a key role in crafting personalized experiences, generating customized content, recommendations, and search outcomes. Natural language processing is another tool at our disposal, facilitating sentiment analysis to interpret user satisfaction through their feedback and reviews. We maintain vigilance over our products through continuous monitoring via analytics tools and dashboards, which is essential in swiftly identifying and addressing any issues or irregularities to uphold product reliability.
Crucial to this entire process is the synergy between data scientists and cross-functional teams. Effective communication and collaboration are vital for turning data insights into tangible product enhancements. The inherent iterative nature of our approach ensures that we continuously evolve and refine our products, aspiring to not only meet but surpass user expectations.
Every product has its success story told through data. In your eyes, what are the key metrics that really tell you a data-driven product is succeeding?
KPIs are tailored to the specific challenges we aim to address, but generally, as a business, we focus on boosting conversion rates, average order value, purchase frequency, and minimizing cost per order, among others. These KPIs are central to our strategy, regardless of the product improvements we initiate.
For more targeted product changes, we concentrate on the relevant metrics. For instance, when altering dasher routes, delivery times become the key indicator. Order quality is paramount when evaluating fulfillment effectiveness, while item/store ranking takes precedence in enhancing discoverability. User order ratings become particularly significant when analyzing customer sentiments.
Typically, there’s a primary metric that we aim to improve, accompanied by a guardrail metric that we monitor to prevent undesirable shifts. For example, if we offer customers a discount on delayed orders to enhance satisfaction, our primary metric would be customer satisfaction scores. Concurrently, our guardrail would be average delivery times, as we need to ensure that the system isn’t exploited, with users deliberately choosing items prone to delays.
Privacy is huge these days. How do you find the middle ground between collecting data to improve products and keeping users’ trust and privacy intact?
As data scientists, we are acutely aware of the need to balance the gathering of data for product enhancement with the imperative of maintaining user trust and privacy. To safeguard sensitive information, we commit to best practices in data anonymization and encryption. Our approach to data collection is minimalist, focusing solely on what is necessary for product development and maintaining transparency about our data use policies.
In place of personal data, we frequently use anonymized user identifiers, thereby reducing risk exposure. We have also established consent mechanisms for data collection, giving users the power to manage their own data through clear opt-in options. Privacy impact assessments are integral to our process, aiding us in identifying and addressing potential risks proactively.
We uphold strict data retention policies to minimize the duration of data storage, and we conduct regular audits to ensure our practices are in compliance. Our culture prioritizes privacy, with comprehensive training for team members on data protection protocols. Additionally, we are proactive in our engagement with regulatory bodies and adhere to industry standards, continuously aligning our data management techniques with the latest in privacy regulations. This commitment enables us to derive meaningful insights while simultaneously securing user trust and ensuring their privacy.
Let’s talk about “Insider Training.” How does data science come into play when you’re crafting the mock interview experience? Are there analytics involved to customize it for each user?
Absolutely. We are leveraging data science within our scope, even though our utilization may not be as extensive as that of larger firms. We apply data science in several key ways.
We have developed an in-house algorithm for pairing candidates with the appropriate interviewer. This system takes into account user preferences, the skillsets of the interviewer, and their feedback to create an optimal match.
We track and update the scores from each interview, enabling candidates to see an average and median score for each round and overall. This provides them with a clear perspective on their performance relative to their peers.
Perhaps the most significant application of data science for us is in interview scheduling. Here, we utilize the availability times provided by candidates and the guidelines from interviewers to generate a conflict-free interview schedule. This is particularly crucial when interviewers are handling numerous interviews in a week.
Feedback loops are vital for product growth. Can you break down how you’ve used data analytics to keep improving, maybe share a story where this made a real difference in a product?
Product enhancement is indeed an iterative process, where gradual improvements are made until resources are depleted. Data science serves as a critical aid to the Product team throughout this journey, providing insights into the effects—both positive and negative—of each iterative change.
Data scientists collaborate closely with Product Managers to evaluate whether specific goals have been achieved without any unintended consequences, as highlighted in my previous explanation about KPIs. When data scientists uncover new insights with each iteration, they inform the product team, which in turn adjusts the product, setting the stage for another round of testing.
A case in point from my experience at DoorDash involved a project on promotional offerings. We introduced a promotion where users could have their delivery fee waived and service fee reduced to 5% if they increased their basket to $80 or more. The promotional messaging was “Enjoy $0 delivery and only a 5% service fee for baskets over $80.”
This promotion was well-received by our regular users, who typically faced $3 in delivery and 15% in service fees. However, we observed a decrease in conversions among our DashPass users (who pay a monthly fee for $0 delivery and reduced service fees). Further investigation revealed that DashPass users were misinterpreting the promotion, believing that their fees would increase because many assumed their service fees were already at 0%, and the promotion’s mention of a 5% fee discouraged them.
We responded by revising the message to “$0 delivery and a reduced service fee.” While this new phrasing resonated with the DashPass audience, it now underperformed among regular users who were unable to perceive the benefit without quantifiable savings.
The solution came with the introduction of two distinct promotional messages tailored to the respective audiences. This targeted communication strategy finally achieved the desired success.
The tech world doesn’t stand still, especially not for data science. Looking ahead, where do you think its role in product development is heading? Any trends or new tech on the horizon that you’re particularly excited about?
The trajectory of data science in product development is on the cusp of considerable expansion and change in the upcoming five years. It will continue to propel innovation, critically influencing the molding of products and user experiences. My enthusiasm is particularly piqued by the following prospects:
- AI-Powered Products: I anticipate AI and machine learning will become even more ingrained, giving rise to products that are capable of providing heightened personalization and increased automation.
- Edge Computing: Data science is poised to broaden its horizon to edge computing, enabling instantaneous data processing and decision-making on IoT and edge devices, which is crucial for real-time applications.
- Data Privacy: Amid intensifying data privacy concerns, I am hopeful that data science will evolve to embrace privacy-enhancing methodologies, such as federated learning, which could revolutionize the way user data is handled while still gleaning valuable insights.
- Explainable AI: This is a domain close to my heart. The development of AI models that are not just accurate but also comprehensible is crucial. Explainable AI can help build trust and credibility in AI-powered products by making their decisions and processes transparent and understandable to users.
IF: Thank you so much for taking the time to share your insights with us today. It’s been incredibly enlightening to hear about your experiences and perspectives on the dynamic intersection of data science and product development. I’m sure our readers will find your experiences at eBay, Amazon, and DoorDash, as well as your work with “Insider Training,” to be as fascinating as I did.
It’s conversations like these that keep our industry vibrant and continually pushing forward.
Takeaways
Abhishek Chaurasiya’s journey is a testament to the pivotal role of data science in sparking product innovation and streamlining efficiency, with AI at the helm crafting a tailored experience for each user. His passion for education is more than just a commitment—it’s a mission to equip a burgeoning workforce with the tools they’ll need to navigate the tech world’s swift currents.
Looking ahead, Abhishek’s insights paint a picture of a data science landscape on the cusp of a riveting expansion. He envisions a future where data-driven products are not just smart but also deeply attuned to the user’s needs—predicting, adapting, and evolving. As he puts it, “We’re not just solving complex problems; we’re anticipating them, making every interaction more intuitive and personal.” With leaders like him at the forefront, the realm of data science isn’t just bright; it’s dazzling with promise.
