EnableEdge at BrowserStack QA Meetup

Speakers and Presentation

The event was graced by the presence of two experienced professionals in the field of QA, viz. Mr. Sachin Kinare (QE Engineer at Orion Innovations) and Ms. Nithya Mani (Lead Customer Engineer at BrowserStack) who provided their own unique and valuable insights on AI testing and how AI agents are being utilized to assist QA professionals to lighten their workload.

Session 1

The first talk, led by Mr. Sachin Kinare, was focused on considerations and pointers to keep in mind as a tester while testing an AI application. Discussed on how testing AI systems goes beyond regular validation. He talked about key testing areas to keep in mind while testing an LLM:

1. Summarization: The ability of an AI to summarize a large amount of data.
2. Categorization: Check if LLM can classify the theme or topic of a user’s prompt.
3. Auditability: Tax paying systems where accountability and accuracy is critical.
4. Accuracy: Data must be relevant to the user and shouldn’t be outdated(RAG or Fine-tuning LLMs can help improve accuracy)
5. Tone and Alignment: LLM responses must be friendly without any form of hateful or inappropriate messages)
6. Regression Testing: To ensure LLMs do not degrade in performance with newer updates
7. Security: Malicious end user can try to hack a prompt and retrieve sensitive information.
8. Configuration: LLMs configuration patterns like temperature, k-factor and token limits affect performance

Mr. Sachin also demonstrated on how libraries such as DeepEval can be utilized to test LLM’s responses against labelled prompts using metrics like GEval.

Session 2

Ms. Nithya Mani introduced the audience to AI-powered testing agents developed by BrowserStack. She explained how these agents help testers automate repetitive work and improve test efficiency. The Test Management Agent takes in project requirements or Jira stories and automatically generates relevant test cases.

It can also identify duplicate or similar test cases and suggest merging or archiving them to keep repositories clean and efficient.

We also got the opportunity to connect with various QA professionals in the field of QA through the meetup. Conversing with them about common interests and passions in the field of technology, listening to their recent works, struggles and how they overcame them was really inspiring and empowering to listen to. We got to learn about new technologies and products created by Organizations and BrowserStack, which helped us to broaden our horizons to the limits of creativity.

Changing AI Landscape

Rapid evolution of the AI landscape has become clear that AI is not just limited to enhancement of existing systems, but it is fundamentally transforming how software is designed, developed and tested. From automated decision making to natural language reasoning, AI systems introduce new forms of complexity a traditional system does not fully address.

The meetup made us realise how important it is for QA professionals to understand not only how AI models work but also how they fail and behave unpredictably. Going forward, we are committed to strengthen our skills in evaluating and validating LLMs by exploring frameworks like DeepVal, LangSmith, LangChain and learn how to design test cases for generative behavior and understanding metrics such as accuracy, hallucination rate and alignment. This is a space we are excited to grow into, because the future of test engineering will not be limited to just software.

To learn on how to test LLMs this will require a shift from traditional pass or fail validation to evaluating behavior, reasoning patterns, and output quality. Unlike regular software systems with fixed logic, LLMs behave probabilistically, where testers must develop new skills such as designing prompt-based test scenarios, measuring hallucinations, measuring factual accuracy, and validating tone and alignment. Modern evaluation frameworks such as DeepEval, LangSmith, and Promptfoo provide structured ways to benchmark LLM outputs using metrics such as correctness, relevance, similarity, bias, and safety. By studying real-world failure cases, experimenting with prompt engineering, and understanding model configurations like temperature and token limits, testers can build the intuition needed to evaluate AI systems effectively. This new domain of testing blends QA fundamentals with AI literacy which makes it challenging.

How can we commit to learn LLM Testing?

The browserstack meetup gave us insights that mastering LLM testing is not optional, it is essential for staying relevant in a future where AI-driven systems will be the norm, we are committed to developing strong AI testing capabilities by continuously experimenting with open-source LLMs, building small evaluation projects, and applying structured testing methodologies to understand their behavior. We plan to practice using evaluation libraries, study industry case studies, follow research from leading AI organizations, and participate regularly in community discussions around AI quality and safety. More importantly, we aim to incorporate AI-specific testing routines into our internal workflows at Enable Edge, so we gain hands-on experience rather than relying only on theory. This long-term approach will allow us to grow alongside the technology and ensure that we remain equipped to test the next generation of intelligent systems.

Conclusion

After attending the BrowserStack QA Meetup it broadened our understanding of how rapidly the world of quality engineering is evolving. The insights shared by the speakers, coupled with conversations with professionals, showcased the importance of embracing AI not just as a tool but as a core skill set for the future of software testing. As AI systems grow more capable and complex, so must our approaches to validating their reliability, safety, and alignment.

This meetup served as a reminder that continuous learning, community collaboration, and hands-on experimentation are essential for staying relevant in this new era of AI. We return with renewed motivation to explore advanced LLM evaluation techniques, adopt modern testing frameworks, and integrate AI-focused QA practices into our workflow. With a commitment to growth and a passion for innovation, we are excited to contribute to a future where quality engineering plays a critical role in shaping trustworthy AI systems.