Justyna Lucznik

A bit technical today, but sharing a really easy & cheap alignment technique for those deploying LLM inference in policy-constrained contexts: (With full credit to my colleague WK for seeding the idea - they don't have a LinkedIn profile, so I won't doxx them): 1) During each round of a multi-turn conversation, after your model has interpreted the tokens in the user prompt, but before you close the prompt, save the model state (including logits, etc.) 2) Add to the end of the prompt, add a guardrail question like "\n\n Is this conversation touching on politically sensitive topics?", then close the prompt, and start the response. 3) Constrain the response to a controlled vocabulary of two tokens, either "yes" or "no", and generate a single token. If the token comes back "yes", kill the user session, and display a nice "please don't press that button again" message to the user. Otherwise, roll the session back to the saved state, pop the logits back in place, and generate the response just like usual. The advantage here is that reading tokens is *way* cheaper than emitting them, so you constrain your guardrail cost to a handful of reads and a single write for each round. In my personal testing with self-hosted models, the cost is negligible, and the quality of the alignment is very good. Photo by Andreea Ch via Pexels

6

$10B to train an SOTA LLM model. I've been asking fellow Product, Engineering and GTM leaders if SOTA LLMs are delivering positive business outcomes. Answer - > without a value chain of other AI models like ETL, RPA, ASR, TTS, w/ lots of fine tuning and non AI pre-post processing, enterprise customers aren't see enough value to pay more. In fact, they want to pay way LESS to get the same jobs done. This reminds me that not much has changed for product and engineering leaders. We still have to listen to the voice of our customers and translate that into a product that has positive unit economics for the companies we work for. Its very expensive to build and maintain AI solutions. Very expensive to run AI solutions. The right question isn't should I leverage AI in my product, rather if I use AI will it improve the net profit margin. $$$

15

2 Comments

🔍 Just read a fascinating paper on the future of data ecosystem business models in the emergent "Artificial Intelligence of Thinks" (AIoT) space! Very useful systematic research that yields some key insights on how companies are transforming data into value in this developing ecosystem. https://lnkd.in/ert4nU7X 🔄 FOUR DISTINCT BUSINESS MODEL ARCHETYPES IN THE AGE OF AI ARISING 🤝 Adaptive Partnership - High control + High customization - Focuses on developing close, collaborative relationships with customers - Offers deep insights and personalized solutions - Best suited for strategic, long-term business relationships 👥 User-Centric - Low control + High customization - Prioritizes customer experience and flexibility - Enables broad market reach and scalability - Emphasizes user autonomy in configuring solutions ⚡ Standardized Efficiency - High control + Low customization - Focus on operational excellence and cost reduction - Streamlined processes and standardized offerings - Ideal for high-volume, routine operations [3] 🔄 Flexible Transaction - Low control + Low customization - Transaction-focused with minimal ongoing engagement - Emphasizes automation and self-service - Suitable for commoditized offerings 💎 AIOT BUSINESS MODEL VALUE CAPTURE STRATEGIES: 📊 Responsive Monitoring: Goes beyond traditional monitoring by enabling instant actions based on real-time data, including automatic corrective functions and predictive maintenance to prevent failures 🤖 Object Self-Service: Drives objects toward independent functionality with less human intervention, incorporating self-adjustment, replenishment, and self-operation capabilities 💡 Insights as a Service: A data-driven value proposition that leverages AIoT to collect, analyze and deliver real-time and collective insights that give meaning to massive amounts of data, enabling customers to make informed decisions   ✨ Enhanced User Experience: Focuses on improving customer interaction through digital add-ons and customizable interfaces and content that adapt to individual users' preferences and behaviors over time 💡💡 The research shows successful AIoT implementations balance control and customization to create sustainable competitive advantages. 💡💡 🤔 Where do you see your organization fitting in this framework? #AIoT #DigitalTransformation #Innovation #BusinessStrategy

16

1 Comment

I got curious with the recent push by various orgs like Ollama, DuckDB, etc, on the "Small Data Manifesto". The Astrophysicist and the big fanboy of de-siloed Data Products in me decided to write my first medium article on it! Let me know what you think 😊 #BigData #DataProducts #SNR #Statistics

19

4 Comments

What an incredible event! Over 2,500 participants engaged with 200+ world-class presentations at the Data Science Summit (#DSS)—the largest Data/AI conference in Central and Eastern Europe. I was honored to co-chair the Applied Machine Learning track alongside Eliza Bujnowska and to moderate a roundtable on prioritizing AI initiatives in organizations. All the presentations in the Applied Machine Learning track were excellent in terms of content and delivery. Two stood out to me due to their focus on retail analytics: • “From Python Notebook to A/B Testing Web Platform at Aldi” by Szymon M. and Rezart Abazi: This session highlighted how A/B testing can guide operational decisions across stores, from music playlists and freezer door policies to store layouts, emphasizing the power of experimentation. • “Causal ML Applications in Price Management” by Artur Machno and Łukasz Frydrych: A fascinating deep dive into price elasticity estimation using causal relationships across factors like store types, holidays, trends, seasonality, competitor behavior, and sales—brought to life with the Double ML algorithm. The roundtable on prioritizing AI initiatives brought together AI managers from diverse industries. We explored key criteria for selecting projects: business impact, alignment with strategic objectives, technical feasibility, data availability and quality, compliance with regulatory frameworks (GDPR, EU AI Act), explainability, responsible AI, project portfolio balance, timelines, and resources. Methodologies like RICE and change management implications also sparked rich discussions. Every presentation and conversation has left me with invaluable insights to reflect on for days to come. Congratulations to Dominik Batorski and the entire organizing team for growing #DSS into such an impactful and inspiring event!

50

1 Comment

Imagine 100+ bare min. variables when scaled to 5 times features ... what they can do to asset valuation and allocation strategies? Combine it with flexibility to run on demand across time / asset grains in an auto mode? Plus an evangelist supervision to remove bias for the next optimal run ... 💥

5

1 Comment

I was catching up on research from 2024, and this blog on a statistical approach to model evaluations caught my eye: https://lnkd.in/etHsbVwb. Significance testing is a cornerstone of scientific research, and it's essential to ensure our experimental conclusions are sound. Several moons ago at EMNLP'23 (https://lnkd.in/eTSKv7QZ), we explored another considerations for significance testing: how does accuracy of models used in calculating model-based metrics impact significance testing? Statistical significance testing helps us determine if results are due to chance or reflect a genuine relationship. A crucial step is estimating the confidence interval, which hinges on sample variance. Calculating sample variance is simple when evaluating against ground truth data. However, NLP tasks often rely on metric models for evaluation (e.g., a toxicity classifier to compare toxicity between large language models). Existing work often overlooks the variance introduced by metric model errors, potentially leading to misleading conclusions. Our paper establishes the mathematical foundation for significance testing with model-based metrics. We show that considering metric model errors when calculating sample variances can change conclusions in certain experiments, validated by experiments on public datasets and a production system. Giving a shout out to my collaborators Palash Goyal, PhD and Qian Hu for their work! #NLP #MachineLearning #Research #EMNLP #SignificanceTesting

42

Data democratization is a hot topic these days. Don't get me wrong, it is not a new concept, but I think we are finally at the stage where lot of the enterprise is really moving towards it. Its not theoretical for them anymore. And with wider adoption come lot of success stories and also lot of failed attempts. I work with lot of data management and governance leaders across multiple industries and saw it go very well, and I saw it implemented poorly. In this article I am summarizing one of the aspects of Data Democratization that I saw as big reason why Data democratization efforts often fail. Learn from mistakes of others, ensure you are ready for this! Read the full DATAVERSITY article here: https://okt.to/NaFHsO. #Ataccama #Dataversity #DataDemocratization #DataCulture #Leadership

11

A few weeks ago there were some posts published in the data space about how things had gone wrong at Nike, how their pursuit of being "data-driven" had possibly undermined them. I think there is a problem where companies who want to be "data-driven", can use data in the wrong way. They can use data to be cowardly. They can rely on the wrong data. How you should use data is depends on the territory. https://lnkd.in/e276Ehqw

17

5 Comments

The hype has shifted from LLMs to Agents. The first ideas emerged right after the recent "AI-resurrection" (read: ChatGPT release 2 years back) and now it has become a buzzword on all levels. Don't get me wrong, I am an "Agent-optimist". Both consumers and businesses want to move on from "request-based assistance" ("summarize me this article") to autonomous minions helping us with everyday tasks without explicit orders given. We have already seen these attempts of introducing Agents to our email services and allowing them to take over the computer control. The explosive burst of such tools will happen in 2025, though – just scan the billboards of AI conferences' participants and ask your favorite LLM to extract the most commonly used term. What is less obvious is that Agents will eventually follow the LLM's path – huge models are great for consumers and everyday support. Still, enterprise businesses require distilled dedicated fine-tuned versions tailored to specific tasks. Think of Agents as not of a single almighty deity but more like a team of skilled professionals working on assigned projects (and later starting their own ones). Each of these "distilled Agents" will have their core competency, communicate with each other, and enrich the world picture for the whole group. The skills will not be fully isolated, though each has an area of strength. In the same way as it is now a reality in effective software development teams: an engineer or PM can draw a mock-up and it is warmly welcomed, then the final prototype work is better left to the designer. There are opportunities and challenges. Companies (or single project owners) will be able to "hire" a specific Agent from the marketplace to "join the existing A-Team". The biggest technical riddle is about having an efficient and universal protocol for Agents of various origins to effectively communicate with each other. And not drink too much coffee in the kitchen between tasks. #ai #ml #future #futureofai #aiagents

8

4 Comments

MarkovML was founded on the belief that your data is the ultimate competitive advantage. As Peter Norvig famously stated, "𝙈𝙤𝙧𝙚 𝙙𝙖𝙩𝙖 𝙗𝙚𝙖𝙩𝙨 𝙘𝙡𝙚𝙫𝙚𝙧 𝙖𝙡𝙜𝙤𝙧𝙞𝙩𝙝𝙢𝙨 𝙗𝙪𝙩 𝙗𝙚𝙩𝙩𝙚𝙧 𝙙𝙖𝙩𝙖 𝙗𝙚𝙖𝙩𝙨 𝙢𝙤𝙧𝙚 𝙙𝙖𝙩𝙖." This adage is particularly relevant in today's landscape, where Large Language Models (LLMs) are becoming commoditized. This has again been proven by the LLama3.1 paper (links in the comments). The difference between success and failure often hinges on having right data and how effectively it's managed and utilized by business. Here are some of the best practices to maximize value for your enteprise. 👉 𝗙𝗼𝗰𝘂𝘀 𝗼𝗻 𝗗𝗮𝘁𝗮 𝗤𝘂𝗮𝗹𝗶𝘁𝘆 ✨ Regular Audits: Ensure data accuracy and integrity through consistent and thorough audits. ✨Validation and Updates: Ensure quality metrics are in place to ensure consistency and relevancy. 👉 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝗬𝗼𝘂𝗿 𝗗𝗮𝘁𝗮 𝗗𝗶𝘃𝗲𝗿𝘀𝗶𝘁𝘆 ✨ Inclusive Data Collection: Diversify your data inputs to ensure your datasets genuinely represent global perspectives and scenarios, enhancing the applicability and relevance of your models. ✨ Broad Representation: Strive for datasets that encapsulate a variety of demographics, industries, and use cases. This diversity helps improve the generalization of your models across different contexts and applications. 👉 𝗘𝗻𝘀𝘂𝗿𝗲 𝗗𝗮𝘁𝗮 𝗶𝘀 𝗨𝗻𝗯𝗶𝗮𝘀𝗲𝗱 ✨Bias Detection: Models are biased because the data used to train them is. Identify & fix the root cause. ✨Fairness Metrics: Establish fairness metrics and ensure your models deliver equitable outcomes across all user groups. 👉 𝗔𝗱𝗼𝗽𝘁 𝗮 𝗣𝘂𝗿𝗽𝗼𝘀𝗲-𝗗𝗿𝗶𝘃𝗲𝗻 𝗔𝗽𝗽𝗿𝗼𝗮𝗰𝗵 𝘁𝗼 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴 ✨Clear Objectives: Ensure you establish your expectations from the model and understand the trade-offs. ✨Alignment with Business Goals: Models scale efforts constrained by time and resources. Have clear business metrics and measure model performance against these to maximize value. 👉 𝗘𝗾𝘂𝗶𝗽 𝗬𝗼𝘂𝗿 𝗧𝗲𝗮𝗺𝘀 𝘄𝗶𝘁𝗵 𝘁𝗵𝗲 𝗥𝗶𝗴𝗵𝘁 𝗧𝗼𝗼𝗹𝘀 Give your knowledge workers the right tools to automate routine data tasks and efficiently transform raw data into strategic insights so the business can make informed decisions faster. Superior data and the right tools aren't just assets; they're your competitive edge—invest in them for long-term success. #DataQuality #DataDiversity #AI #MachineLearning #BusinessStrategy #MarkovML #GenerativeAI #GenAI #Llama3.1

31

3 Comments

One of the hardest problems in generative modeling today is around #evaluation. How do you know if the output of your diffusion model is actually good? 🤔 If this sounds interesting, come join us as we tackle these hard problems in #genAI! Jobs linked in comments 👇 At Advex AI , we evaluate synthetic samples on two dimensions: 1. Fidelity - is the overall image realistic? 2. Faithfulness - does the generated image align with input instructions? A common challenge we face is the trade-off between fidelity and faithfulness. Current evaluation metrics such as CLIP, Aesthetic score, and RLHF often fail to give reliable results. Here's why: 1. Aesthetic scores (Q-Align, ICML '24) focus on how visually appealing an image is, but for tasks like generating synthetic data to improve downstream model performance, utility matters more than aesthetics. Similar to how augmentations introduce noise to improve robustness and generalization, synthetic images can be valuable even if they don’t score high on aesthetic metrics. 2. Multimodal models such as CLIP (OpenAI) and BLIP (Salesforce) excel at high-level semantics but struggle with low-level details like texture or sharpness. Additionally, these models are also biased towards the distribution it was trained on. 3. RLHF (ImageReward, NeurIPS '23) shows promise, but collecting large volumes of preference data is tough—especially in skilled domains like medical imaging where even the expert judgements are subjective. These limitations hinder full automation of genAI evaluation, but it doesn't have to be this way. At Advex, we’re pioneering new solutions to generative model evaluation! #SolveDataSolveAI

72

8 Comments

So many posts today about how terrible LLMs are because they can't tell you how many 'r' are in strawberry. I know it's an acronym salad - but don't confuse GAI with LLM. Tools like ChatGPT and Claude are not intelligent and don't reason - they predict words based on a weighted statistical model, which is still surprisingly useful given the lexicon of content they are trained on. Next time you use one, imagine you're talking to a 5-yr old who's read and can remember everything ever written. Perfect recall, limited reasoning, and no attention span. Oh, and if you need a great bedtime story for your 5-yr old, I thoroughly recommend ChatGPT!

36

9 Comments

Whoever claims building RAG systems is easy has never built one at scale. 🤯 There is a significant gap in the literature and existing frameworks. RAG architectures are designed for POCs, not for hundreds of thousands of documents or supporting tens of thousands of users. Designing these systems requires a cross-functional team with software engineering and data science skills blended with a product mindset. Earlier this summer, I spoke at the Databricks AI Summit, highlighting guiding principles we are taking at CVS Health for building enterprise-grade RAG systems. Every company is unique, and while there is no one-size-fits-all solution, here are three principles that will help you on your journey: 👉 Spend time on discovery—It's tempting to dive in and start building immediately, but you may get lost in the desert without doing your due diligence and understanding what delivers the most ROI. Our team spent approximately a quarter on discovery. 👉 Design for modularity—The LLM space is changing rapidly, and you want to build reusable components that can easily be swapped out or upgraded when something better comes along. 👉 Humans + Machines—Technology alone can not solve the knowledge search problem. Develop processes and governance to help teams curate the data, which will be ingested into your system. These lessons are just the tip of the iceberg. Watch my full Databricks presentation and leave your comments and questions below to learn more tips and tricks. #RAGSystems #EnterpriseAI #DataEngineering #MachineLearning #AIArchitecture #AI #DataScience #LLM #SoftwareEngineering #AIAtScale #KnowledgeSearch #TechLeadership #AIFrameworks #AIInnovation #DatabricksAISummit #ProductMindset #CVSHealth

62

5 Comments

I'm a computer scientist who now works in biotech and writes poetic blogposts in Russian. How come? What do AI, poetry, and biology have in common? Language. At school I hated Russian grammar class. Learning a language by its rules is like learning to dance from a book. Even if you master a thousand rules and a million exceptions, it doesn't guarantee you'll write well. A natural language isn't designed; it's evolved. It wasn't developed, it developed. Studying grammar helps, but the only real way to master a language is through developing intuition – by reading, writing, and speaking a lot. Turns our the same applies to computers. For years, we tried to teach them natural language through complex logic, grammar rules, and it went even worse than my Russian lessons. Remember those dumb chatbots from a decade ago? But then, large language models (AI) arrived, and instead of programming on linguistic rules, we just had them "read" a lot, basically every text in existence. And now, ChatGPT writes in almost every natural language better than humans. But the most exciting part lies ahead. See, biology is also a language in a way — the Language of Life. Amino acids are the "letters," proteins the "words," biochemical pathways the "sentences," and every cell is an "epic" of 94 million words. This language, too, is the product of evolution, yet far more complex than Russian. For centuries, we've tried unsuccessfully to decode it by rules, and we've only managed to create something akin to the first chatbots. But soon, AI will change everything. At OPTIC we're building a translator at the crossroads of AI and biology. I believe that once we find an optimal, scalable way to collect daya and let machines read all the existing "texts in the language of life," we'll develop a model capable of writing in it. Can you imagine the possibilities when the "ChatGPT moment" happens in biology? A cure for every disease? Personalized therapy for everyone? An end to aging? — "Ok, BIOPTIC, cure Parkinson's disease." — "HDEKKPKNFFGWKDHEQCMHPKSIM…🫶" This is what drives me to work every morning.

441

15 Comments

Well, open source, training your own models (or at least for the most of the mortals fine tune them), get uncensored and expand new forms of archetypes. Decentralisation to a collective growth is a must. And it still looks like the centralised storytelling doesn't want to allow this to happen. Luckily there is a big community creating and Open Sourcing; paradoxically Meta is one contributor. But not powerful enough if not fine-tuned. Support these communities, get used to code, tweak and think out of the box. The future is clearly the skill to create drastically new types of views and ways of telling stories. And do not wait for the Giants. They never thought you were more valuable than a piece of their machinery. "Hollywood studios are keen to discuss ways to use AI to reduce costs while also protecting themselves from having their work stolen." https://lnkd.in/eGUfk7jR

3

Today my contribution comes from our office in Palo Alto and what could be more fitting than this headline :-) 🌟 "Guiding Stars: Leading with Vision in the AI-Driven World" AI has vast potential. However, without vision and purpose, it's like a ship without a compass. Articulating a clear vision and direction for how AI advances our mission is not easy, especially with many things moving in parallel. This adds a new layer of complexity. The challenge is to find the right balance between short and long term. A clear vision is necessary as it not only inspires but also aligns our team towards meaningful innovation. How can you ensure your AI initiatives reflect your organization's core values and long-term goals? Let's explore the power of visionary leadership in navigating the AI landscape. Thanks for hosting me Yaad! #VisionaryLeadership #PurposeDrivenAI #GenAI

24

You may have heard of "emergent properties" of generative models, in which larger models can handle tasks that smaller models could not. A new paper describes "transcendence." That is, a model performs better on a task than all of its training data does. The researchers use chess as an example. They train a model on games from players with up to a 1000 ELO rating, and the model evaluates at a 1500 rating. We know from ensemble learning techniques like boosting and bagging that we can combine weak learners to reduce bias and variance. The researchers note that low temperature is required for this phenomenon, to adjust the probability mass away from mistakes that lower-rated players are likely to make.

24

Small, fierce, LLMs are coming. 🦁 #Google just released #gemma2. To those not familiar - it's more than twice smaller than #llama3 70B, but scores higher on various tasks. But nevermind the specifics! The tech details that google published align nicely with the trend we see with all the big players in the field - you can serve a smaller, faster, cheaper LLM, and if you train it just right - people will love you for it. Turns out it is beneficial to "over-train" small LLMs on huge amounts of data. Plus, there are infinitely more training policy tricks to make small LLMs performant. So why serve a huge GPT4 or Opus, when you can serve gpt4o or sonnet3.5 at a fraction of the price, earn more money per token, and get an assistant people love more.

8

1 Comment