When your model learns more than it should There is a quiet assumption in most machine learning projects. If the model is accurate, then it must be good. That assumption works well, until the model starts making decisions about people, money, risk, or opportunity. Then accuracy alone is not enough. A model can be very accurate and still systematically treat groups differently. This is the ninth article in our series "The Theory behind Qlik Predict". So far, we have talked a

Author: Igor Alcantara A deep dive into Talend Data Quality’s matching magic, from T-Swoosh to Survivorship If you have been following my series on the theory behind Qlik, you know we usually spend our time analyzing data that is already nicely polished. We look at averages, distributions, machine learning, LLMs, RAG, and charts. However, every developer knows the dark secret of our industry, which is that data is rarely polished when it arrives at our doorstep. It is usually

Author: Igor Alcantara Picture yourself surveying a city from above. Each bustling building, large avenue, and hidden courtyard tells a story, but what if you had to sum up the entire city in a single snapshot? This is the power, and the profound puzzle, of Measures of Center Tendency . These are the tools that shrink data chaos into a single, meaningful beacon. They are critical, sometimes cunning methods for finding patterns in the numbers that shape your business, your str

Author: Igor Alcantara Close your eyes and think of “apple.” What popped into your mind? A crisp fruit? Maybe the company with the bitten logo? Maybe your grandmother’s pie, or Steve Jobs in a black turtleneck? In my case, the Beatle's recording label after they left Parlophone? That chain reaction, one idea sparking a web of others; is how the human brain works. Neuroscientists call it associative memory : our neurons form networks where one concept lights up another. When w

Author: Igor Alcantara Every dataset has a bit of Dr. Jekyll and Mr. Hyde in it. On the surface, it might look like one thing, but underneath it can take on multiple identities depending on how you examine it. In fact, sometimes it’s not just two identities; it’s ten. That’s where classification comes in. Unlike regression, my previous article, which predicts how much or how many, classification sorts data into categories, revealing its hidden personas. Sometimes the split i

Q(p) = Regression - Headache Author: Igor Alcantara When someone says “regression,” your brain might first jump to childhood flashbacks of questionable fashion choices or to the time you fell back into eating instant noodles for dinner three nights in a row. Or maybe regression is the haircut I used in the 1990's when I was the singer and lead guitar player in a rock band (see below). But in the world of data science, regression is about moving forward using past data to pred

Real photo of Igor Alcantara in a regular ML project Author: Igor Alcantara There is a common trend in the world of data that is a bit dangerous. A person watches a couple of YouTube videos, learns how to use some no-code tools, Auto ML type or RAG, and claims "I am a data scientist" or learn a few prompts and say they are an "AI Expert". What bothers me is not the title itself. As I explore in a previous article , titles only matter if you do not see beyond the words. What c

Retrieval-Augmented Generation and the Layers Beneath Your Data Author: Igor Alcantara What if your enterprise data had a subconscious? A deep, multilayered repository of memories. Some crisp and current, others buried in old documents or silos, waiting to be awakened. What if, instead of going through dashboards and PDFs, you could simply ask a question and get an intelligent, verified answer? Welcome to the world of Retrieval-Augmented Generation, or RAG, and welcome to the

Author: Igor Alcantara Imagine your data as a cluster of clocks orbiting a massive gravity mass: sales flowing by the minute, inventory pulsing in discrete beats, marketing spend stretching and contracting like spacetime around the edge of a black-hole. On their own, each clock tells a fragment of the story; together, their shifting rhythms bend forecasts the way relativity bends light, revealing hidden paths through uncertainty. Multivariate time-series forecasting captures

Author: Igor Alcantara Let's board our Delorean and travel back in time a little. Not to 1955 or even 1985 but just a few months: April 2024. Back then, I started a series of articles aimed to explain the theory behind Qlik Predict . I started explaining the explanation of Machine Learning predictions. In other words, my first article in this endeavor was about SHAP Statistics . A few months later, I wrote about Qlik Predict Preprocessing tasks, and my very last article of 2

The image says Qlik AutoML because this was the name of Qlik Predict when this article was written Author: Igor Alcantara Welcome back, fellow data adventurers! If you've been journeying with us, you've already navigated the treacherous terrains of preprocessing tasks in Qlik Predict , and unlocked the enigmatic secrets of SHAP values and feature importance . There articles are part of my series where I explain the theory behind Qlik Predict . I hope you are a ready for the t

Author: Igor Alcantara We take a lot of things for granted. For those new to Machine Learning who are lucky to start with tools like Qlik Predict , there are a lot of things happening behind the scenes that veterans (aka classical practitioners) like me had to program in R and Python in the past, just like the old Aztecs did (well, not really, but I like the joke). Don't get me wrong - this is amazing. We can now focus more on the analytical and scientific parts of the work r

Author: Igor Alcantara On October 15th, 2024, a user in Qlik Community (Martin) asked a very interesting question: what is the exact mathematical formula for Skewness and Kurtosis in Qlik? Check the original post here . Since that information is not available in Qlik Documentation , I decided to do some reverse engineering. My approach was to use a technology where I have the mathematical documentation about it, test it there, and compare it to the numbers I got in Qlik. Wha

Hello, Data Explorers! 🚀 In this video, we dive into the practical applications of Correlation and Mutual Information within Qlik. Discover how to leverage these powerful tools to uncover hidden relationships and insights in your data. Whether you're analyzing linear trends or complex dependencies, this tutorial will guide you through the process step by step, helping you make more informed decisions with your data. 👍 If you appreciate this video, like it and share it. 🔍 F

Author: Igor Alcantara Have you ever wondered how economists predict market trends, or how biologists understand the relationship between environmental factors and species behavior? This is where regression analysis comes into play, a cornerstone in the world of statistics with applications spanning finance, economics, biology, and social sciences. It's a method that dives deep into the dynamics between variables, offering insights into how changes in one or more independent

Author: Igor Alcantara In today's data-centric landscape, much fanfare surrounds the predictive prowess of machine learning, often casting it as the beacon of the digital age. Yet, while prediction is undeniably a potent facet, the true crown jewel of machine learning lies in its prescriptive capabilities. Moving beyond simply forecasting future events, the power of prescriptive analysis provides actionable insights and recommendations, enabling organizations to not just fore

Author: Igor Alcantara Bayesian statistics sounds fancy and intimidating, like something you need a tuxedo, a pipe, and a PhD to understand. But at its core, it’s not that complicated. It’s basically a way of saying: “Here’s what I thought before, here’s the new evidence I just got, so now here’s what I think.” It’s how humans have operated since forever. If you thought it was going to rain today because the sky looked gray, but then you check your weather app and see 0% chan