AI Trailblazer

"AI serves as a catalyst for personal and professional growth in the corporate sector by empowering individuals to leverage technology to its fullest potential, drive business success, and shape the future of work"

-Shashank Shekhar, Principal Ordinate, AIOrdinate

With the dawn of Artificial Intelligence (AI), individuals are harnessing its potential to amplify their productivity and creativity. Prominent language models such as GPT-4, Mistral, Llama, and Gemma are making waves on a global scale, driving the democratisation of AI from a consumption perspective. Open-source libraries serve as pivotal tools in achieving optimal outcomes.

With over 15 years of extensive experience in AI for both B2C and B2B applications, Shashank Shekhar, Principal Ordinate, AIOrdinate, stands as a trailblazer in Artificial Intelligence and Machine Learning. His contributions have significantly impacted numerous large tech companies, including Amazon, VMware, Flipkart, Target, and at Subex, as Head of Global AI Practice. Shashank has pioneered cutting-edge industry-first products and solutions, filed multiple patents, published papers in top ABDC listed journals, and created several open-source libraries, utilised by a vast community of AI practitioners worldwide.

In an exclusive interview with Corporate Citizen, Shashank delves into the intricacies of the AI landscape, its profound impact, and the perpetual journey of learning. Let us take a closer look at the fascinating world of artificial intelligence

Corporate Citizen: Tell us about your current role as Principal Ordinate?

Shashank Shekhar: In my current capacity as Principal Ordinate at AIOrdinate, I oversee both the consulting and product verticals. AIOrdinate stands as a distinguished product and consulting firm operating within the Generative AI domain, catering to a diverse array of industry verticals such as healthcare, fintech, e-commerce, and retail. Backed by a seasoned team of AI professionals with a proven track record, we specialise in delivering transformative solutions tailored to meet the unique objectives of our clients. Through a collaborative approach, we forge strong partnerships, ensuring mutual success.

Our vision at AIOrdinate is centred on elevating enterprise businesses from operational efficiency to strategic excellence, crafting bespoke results-driven solutions by harnessing the power of AI and machine learning. Additionally, we are pioneering the development of an industry-first revolutionary Large Language Model Operations (LLM Ops engine), facilitating the deployment of LLMs at scale.

At AIOrdinate, we prioritise privacy and data security in our deployment of LLM-based applications and products, safeguarding high-value organisational data through on-premise or private cloud solutions. Leveraging proprietary smaller LLMs and Retrieval Augmented Generation (RAG) pipelines, our approach enables efficient training and deployment on small-scale infrastructure while offering tailored solutions to meet industry-specific requirements.

With over eight years of deep learning expertise and built-in privacy measures, our approach ensures superior performance on specific tasks compared to existing models like GPT-4 or Gemini Pro/Ultra, at a reduced deployment cost and timeframe.

In addition to my leadership role, I am deeply involved in various aspects of AIOrdinate's operations, including talent acquisition and development, defining and executing product and solutions strategy, managing existing customer relationships, ensuring customer success, and spearheading new customer acquisition efforts.

CC: What were the obstacles you faced while building a team?

Building a team presents a myriad of challenges, spanning recruitment, development, and garnering management approval. One of the critical aspects is demonstrating tangible value and return on investment (ROI) through our work to secure necessary funding and endorsement from management. Acquiring top-tier talent proves to be a significant hurdle, particularly within the competitive landscape of data science. This challenge is further accentuated in smaller or newer companies, such as startups.

Prior to joining Subex, I had the privilege of working with esteemed global brands, where attracting top talent was relatively less daunting. However, upon joining Subex, which was not yet established as a top AI company, attracting premier talent from academia and industry presented a distinct challenge.

To overcome these obstacles at Subex, we implemented targeted strategies. Initially, our focus was on recruiting promising graduates and providing them with comprehensive training to groom them into proficient data scientists. Concurrently, we invested substantially in bolstering Subex's brand presence within the industry. This involved active participation in conferences, delivering influential talks, publishing impactful papers in esteemed journals, filing patents, and engaging with colleges to attract top-tier talent. These branding initiatives played a pivotal role in our success.

Throughout my tenure, Subex emerged as a prominent player in the data sciences arena, earning recognition as the 15th best company for data scientists and ranking third in terms of productivity and engagement. This achievement was the culmination of concerted efforts across various fronts, including branding and talent development. Additionally, we emphasised hands-on involvement in cutting-edge industry projects to underscore the value of our team, both internally and externally. A significant focus was placed on intellectual property (IP) contribution, manifesting in the publication of high-quality papers and invention disclosures. Furthermore, we fostered an environment of intellectual freedom, ensuring that team members,regardless of experience, were afforded ample opportunities for experimentation and ownership. Moreover, we placed a strong emphasis on recruiting top talent globally to augment our capabilities and solidify our position in the industry.

At AIOrdinate as well, we prioritise the recruitment of top talent, recognising its paramount importance, particularly in our early-stage startup environment. We understand the significance of sourcing individuals who are inherently driven and possess a keen appetite for learning and growth. To this end, we have taken proactive steps to cultivate a team of exceptional professionals.

Our strategy includes recruiting our first cohort of interns from esteemed institutions such as Narsee Monjee Institute of Management Studies. (NMIMS) and Indian Institutes of Information Technology (IIIT). Moreover, we have developed a cutting-edge methodology aimed at nurturing and developing these interns into some of the finest AI practitioners in the industry. Through this approach, we aim to harness the potential of homegrown talent and foster a culture of innovation and excellence within AIOrdinate.

CC: What are the notable projects you have done in the field of AI?

During my tenure at AIOrdinate, a significant focus has been on developing a framework aimed at dramatically reducing the deployment costs associated with LLMs. We've successfully devised a strategy that enables us to leverage multiple LLMs, including on-premises options, while implementing selection, routing, and hedging based on various factors such as cost, latency, quality, and user feedback and preferences. Additionally, we've implemented Automated Prompting Strategies, including Zero-Shot, Few-Shot, Chain of Thought (CoT), and Ensemble Shot, to enhance efficacy.

"Rather than viewing AI as a threat, we can see it as a tool that complements and expands human creativity, pushing the boundaries of what's possible in the creative field"

Our approach involves a comprehensive four-step process:

1. Prompt selection and adaptation: This phase encompasses various techniques such as prompt shortening, prompt selection, query deduplication, and query concatenation to optimise prompts for cLLM interaction.

2. LLM Approximation: We employ methods like Completion or Response Caching, Model Evaluation, and Benchmarking, as well as Guardrails to approximate LLM outputs efficiently.

3. Composite Request Hedging and Fine Tuning: In this stage, we utilise techniques like RAG, Model Fine Tuning, and User Preference and Feedback analysis to refine LLM responses further.

4. LLM Cascading: Finally, we implement a Scoring Function Generator and LLM Router to manage the cascading of LLM requests effectively.

These efforts have significantly enhanced the cost-effectiveness and performance of deploying LLMs, positioning AIOrdinate as a leader in the field of LLM optimisation and implementation.

During my tenure at Subex, I spearheaded the development of HyperSense, a project I consider among my proudest achievements. This endeavour bridged engineering and AI realms, affording me the opportunity to lead cross functional teams comprising backend developers, frontend developers, development and operations engineers and AI scientists. HyperSense AI stands as an end-to-end AI Orchestration platform, empowering enterprise clients to make quicker, more informed decisions through AI utilisation across the data value chain. Notably, the platform facilitates seamless data aggregation from diverse sources, empowers users—regardless of coding knowledge—to translate data into actionable insights by constructing, interpreting, and refining AI models, and effortlessly disseminate findings throughout the organisation; all within a user-friendly, no-code platform.

At VMware, I spearheaded the development of a groundbreaking list price optimisation model, which relied on a customised decision tree framework. While traditional decision trees typically employ binary splits based on impurity measures like Gini/entropy or mean squared error (for CART), our project adopted a pioneering approach utilising non-binary decision trees. In this innovative framework, the order of splits was determined by business needs, with impurity measures serving as stopping criteria. This novel approach empowered VMware to adjust product prices

Career journey and growth

"By leveraging AI-driven insights and analytics, individuals can make more informed decisions and optimise their workflows,leading to improved performance and career advancement opportunities"

Affinity for numbers and mathematics

Following my graduation, I embarked on my professional journey with Marketics, a distinguished boutique consulting firm renowned for its expertise in analytics. Marketics boasted an impressive clientele, including global giants like Coca-Cola and P&G, setting the stage for the initiation of my career. Despite receiving offers from larger corporations, my passion for quantitative analysis, rooted in my affinity for numbers and mathematics, led me to choose Marketics, where I devoted nearly two years of my professional endeavours.

Marking a pivotal phase of my career

Thereafter, I transitioned to Target, a leading retail powerhouse in the United States. Over the span of four years, I immersed myself in diverse realms, spanning customer insights, CRM operations, merchandising, and research and development (R&D). Towards the culmination of my tenure at Target, I became part of an elite team dedicated to pushing the boundaries of machine learning and crafting state-of-the-art models, marking a pivotal phase of my career.

My journey continued as I ventured into Amazon, commencing my tenure in the inventory planning and control team (IPC), where I spearheaded the development of economic order models. These models, pivotal in decision-making processes, determined optimal procurement strategies, inventory quantities, and storage allocations. Subsequently, in 2013, I transitioned to Amazon India as one of the pioneering members of the analytics team, marking the inception of their analytics and data sciences division, with the distinction of being their inaugural hire in this domain. During my tenure at Amazon India, I played a pivotal role in driving marketing analytics, primarily as a member of their esteemed traffic team. Here, I was instrumental in establishing the entire analytics framework, crafting essential artifacts such as the multi-channel attribution model, traffic anomaly detection model, and channel performance models. Additionally, I spearheaded the development of numerous models, particularly in the domain of affiliate marketing, including the implementation of first touch and last touch attribution models for optimising marketing expenditure.

Following my journey with Amazon, I transitioned to Flipkart, where I assumed leadership of the customer service analytics team. In this capacity, I was tasked with establishing and nurturing the team along with its associated practices. I consider myself privileged to have been at the forefront of the customer service analytics landscape in India, where I led innovative initiatives aimed at enhancing metrics such as First Call Resolution (FCR), One Call Resolution (OCR), and Interactive Voice Response (IVR) routing and optimisation.

Taking a significant turn

Until 2014, my professional journey was predominantly within the realm of business-to-consumer (B2C) environments, with stints at Marketics, Target, Amazon, and Flipkart. However, my career trajectory took a significant turn when I transitioned to VMware from Flipkart. This move represented a pivot towards the dynamic landscape of business-to-business (B2B) high-tech ventures, where I assumed the role of one of the pioneering members of the Data Sciences team at VMware during the nascent stages of its development.

At VMware, I spearheaded the formation of a highly skilled team of AI practitioners, focusing on diverse areas such as pricing, marketing, sales, and research and development. Throughout my tenure, I played a pivotal role in shaping and executing strategies across these domains. Notably, as a founding member of the data sciences team, I orchestrated the expansion of the team from a single location to a global force comprising 16 members across India, Bulgaria, and the USA within a mere six months.

My contributions at VMware were marked by substantial impact, with a testament to my leadership being the generation of a staggering US $65 million through end-to-end project oversight. This included pioneering initiatives such as industry-first list price optimisation and the rationalisation of special discounting practices tailored for the B2B domain, further solidifying VMware's position as a leader in the high-tech sector.

Assuming leadership of the global AI practice

In August 2019, I embarked on a new chapter of my career at Subex, assuming leadership of the global AI practice. At Subex, I undertook the monumental task of building the entire team from the ground up. Witnessing remarkable growth, I expanded the team from a modest six members to an impressive count of over 60 professionals.

My tenure at Subex was characterised by the creation and management of high-performing international teams specialising in data engineering, backend and frontend engineering, DevOps, machine learning and deep learning, across diverse global locations. I personally oversaw all aspects of recruitment, training, coaching, mentoring, and performance development, ensuring that we attracted and retained the best talent from both industry and academia worldwide.

A significant aspect of my role at Subex involved integrating AI into all existing products while simultaneously spearheading the development of AI-first solutions from scratch. I established a cohesive and high-performing team spread across various geographical locations, including Bangalore (India), Dubai (UAE), and Santiago (Chile).

Notably, our efforts were recognised by Analytics India Magazine, ranking Subex as the 15th best team for data scientists to work for and the 3rd best in productivity and engagement. Among our achievements was the conceptualisation and development of HyperSense, an industry-first, state-of-the-art data platform. HyperSense facilitated end-to-end AI orchestration, encompassing ML and DL model building, online streaming, feature engineering, a feature store, a comprehensive auto ML suite, and full-fledged ML Ops and DL Ops capabilities.

About three months ago, I embarked on my latest endeavour as the Principal Ordinate at AIOrdinate. While it's only the beginning, I am brimming with excitement for the boundless possibilities that lie ahead in this role.

effectively, resulting in over $30 million in incremental revenue within the first year alone.

CC: What are your thoughts on open-source library? Share your experience of the two open-source libraries you published.

As previously mentioned, my experience includes constructing AI models, which often function as black boxes—wherein the internal processes remain opaque despite understanding the input and output. This lack of transparency underscores the importance of developing Explainable AI (XAI) models, aiming to shed light on the decision-making processes of AI systems.

One notable contribution to XAI is the development of the dominance analysis library, a game theory-based method designed to elucidate AI models. Collaboratively built with my colleagues Sajan and Siva, this Python-accessible library has garnered widespread adoption, with over 1,50,000 data scientists worldwide leveraging its capabilities to enhance transparency in AI models.

Additionally, we also developed Exploripy, an innovative tool focused on automating exploratory data analysis—a critical aspect of understanding data attributes and characteristics. Unlike traditional approaches solely centred on constructing AI models, Exploripy streamlines the exploratory data analysis process, enabling comprehensive analysis with just two lines of code. This user-friendly tool has been embraced by over 1,00,000 data scientists globally, positioning it as a valuable asset in the data science community.

Our commitment to contributing to the data science community extends beyond the development of these tools. By embracing open-source principles and sharing our expertise, we aim to empower fellow data scientists and foster collaboration, ultimately driving advancements in the field. This dedication to knowledge-sharing and community engagement reflects our ethos of giving back and facilitating collective growth and innovation in data science.

CC: How AI helps an individual to grow in a corporate sector?

AI plays a pivotal role in fostering personal and professional growth within the corporate sector by enhancing efficiency, streamlining processes, and accelerating business outcomes. Enterprises are increasingly embracing AI due to its ability to augment human skills and knowledge, ultimately leading to greater productivity and innovation.

One of the keyways AI contributes to personal growth in the corporate sector is by automating repetitive tasks, allowing employees to focus on higher-value activities that require critical thinking and creativity. By leveraging AI-driven insights and analytics, individuals can make more informed decisions and optimise their workflows, leading to improved performance and career advancement opportunities.

Moreover, AI enables continuous learning and development by providing access to vast amounts of data and resources. Through AI-powered tools and platforms, employees can upskill and reskill themselves in areas such as data analysis, machine learning, and programming, thereby staying relevant and competitive in the ever-evolving corporate landscape.

Furthermore, the adoption of AI fosters a culture of innovation within organisations, encouraging employees to explore new ideas, experiment with emerging technologies, and collaborate on transformative projects. As AI becomes more integrated into various sectors, individuals who possess AI-related skills and expertise are poised to thrive and lead in their respective fields.

Overall, AI serves as a catalyst for personal and professional growth in the corporate sector by empowering individuals to leverage technology to its fullest potential, drive business success, and shape the future of work.

Operating in isolation without awareness of global developments can hinder progress and lead to challenges. Therefore, grasping the dynamics of the ecosystem, identifying beneficial opportunities, and mastering navigation within it are crucial

CC: Will AI significantly disrupt and replace creative work?

AI serves to enhance human creativity within the creative domain. AI undeniably impacts the creative industry by assuming certain creative tasks previously undertaken by humans. What distinguishes the creative realm is its lack of definitive right or wrong answers, in contrast to disciplines like mathematics, where correctness is paramount.

In creative endeavours such as drawing or image crafting, outcomes are not strictly binary, allowing for a spectrum of possibilities. This ambiguity creates an opportunity for generative AI tools to enter the scene, potentially replacing human creators. While AI-driven substitution is indeed emerging in the creative domain, it's crucial to acknowledge that this is just the beginning of AI's influence.

As AI continues to evolve, it offers unprecedented possibilities for collaboration between human creativity and machine intelligence. Rather than viewing AI as a threat, we can see it as a tool that complements and expands human creativity, pushing the boundaries of what's possible in the creative field

CC: What are some of the recent developments in AI and how do you see AI in the near future?

Generative AI, including Large Language Models (LLMs), represents a significant advancement in the field of artificial intelligence. However, the process of independently building and deploying large language models can be prohibitively expensive due to the substantial compute resources required. As a result, there is a growing focus on optimising compute and cost efficiency in AI development.

Efforts are underway to refine existing LLM models to reduce costs and develop smaller, task-specific models, which presents a promising avenue for advancement. Additionally, significant developments are occurring in sectors such as autonomous vehicles, particularly in computer vision and deep learning domains, paving the way for substantial progress and innovation in these areas.

While the pursuit of specialised AI applications continues, there is also significant progress being made in the development of Artificial General Intelligence (AGI). AGI aims to emulate humanlevel intelligence across diverse tasks and domains, offering immense potential for revolutionising various industries.

The implications of achieving AGI are vast and far-reaching, spanning sectors from healthcare to transportation, education to entertainment. AGI has the capability to transform how we interact with technology, enhancing our capabilities and facilitating new forms of collaboration between humans and machines.

It is imperative for individuals to embrace the integration of AI into their workflows and skillsets. Denying the reality of AI's impact is not a viable option, as those who fail to adapt risk being replaced. To ensure job security and remain relevant in an AI-driven world, individuals must cultivate adaptability and effectively leverage AI technologies.

By understanding and harnessing the power of AI, individuals can capitalise on its transformative potential to reshape industries and drive innovation. Dispelling misconceptions about AI's role in job displacement is essential, as efficient utilisation of AI can enhance job security and create new opportunities for growth and advancement. It is crucial for individuals to proactively engage with AI advancements and leverage them optimally to navigate the evolving landscape of work and technology.

CC: What is the most invaluable pieces of advice you have received in your journey in the corporate world?

One of the most invaluable pieces of advice I have ever received is that learning is an ongoing journey. The notion that learning ceases after accumulating a certain amount of experience, say, ten years, is a misconception, especially in dynamic fields like artificial intelligence and beyond. The reality is that constant evolution is inherent in these domains, demanding perpetual adaptation and acquisition of new knowledge.

The consequence of failing to embrace continuous learning is stagnation and eventual obsolescence. Conversely, by consistently expanding one's knowledge base and staying abreast of advancements, individuals ensure their relevance and indispensability in their respective fields. This principle of lifelong learning has been instilled in me by one of my mentors, who emphasised the importance of never ceasing to learn.

The wisdom in this advice is profound. It underscores the necessity of maintaining a curious and inquisitive mindset, always seeking to broaden one's horizons and deepen one's expertise. As I've internalised this lesson, it has become a guiding principle in my professional journey, enabling me to navigate the ever-changing landscape of technology with confidence and agility.

"By embracing open-source principles and sharing our expertise, we aim to empower fellow data scientists and foster collaboration, ultimately driving advancements in the field"

CC: In the 15 years of your career journey, what changes have you witnessed in the working environment?

Over the span of my 15-year career across various companies, I have witnessed significant shifts in the working environment. Primarily, there has been a widespread embrace of technology, with digital tools and artificial intelligence transforming how we work and collaborate. The Covid-19 pandemic accelerated the trend towards remote work, leading many companies to adopt permanent remote work policies, thereby enhancing flexibility and work-life balance.

Furthermore, there has been a notable focus on diversity and inclusion, with companies prioritising efforts to create more inclusive environments and foster diverse teams. Agile methodologies have also gained traction, extending beyond software development to improve collaboration and adaptability in various business areas.

The rise of the gig economy has reshaped employment dynamics, with more individuals opting for freelance and contract work. Additionally, companies are increasingly prioritising employee well-being, implementing initiatives to support mental health and overall wellness.

Remote collaboration tools have seen a surge in adoption, facilitating seamless communication and collaboration among distributed teams. Overall, the evolving working environment underscores the importance of adaptability, flexibility, and a focus on employee well-being to thrive in today's dynamic landscape.

CC: What is your advice for people who are entering into startups or corporate sector, but unable to understand equations of AI?

Understanding the ecosystem in which you operate is paramount to success. Operating in isolation without awareness of global developments can hinder progress and lead to challenges. Therefore, grasping the dynamics of the ecosystem, identifying beneficial opportunities, and mastering navigation within it are crucial.

Prioritising self-improvement involves a balanced approach. While it's essential to address weaknesses and strive for growth in those areas, it's equally vital to nurture and enhance strengths. Neglecting strengths may inadvertently weaken them over time, emphasising the importance of consistent refinement and elevation.

Strategically leveraging strengths while mitigating vulnerabilities is key. By focusing on strengths, individuals can capitalise on their competitive advantages and maintain a strong position within the ecosystem. Simultaneously, addressing weaknesses ensures a well-rounded skill set and minimises potential areas of vulnerability, fostering sustained growth and success.