Hello, welcome everyone. This is the first workshop on a series of workshops on artificial intelligence. So we're going to start with intro to AI. I will warn you that it starts easy, but it gets very difficult very quickly. So make sure you have a notebook and a pencil ready. Out of curiosity, if you guys just want to put it in the chat, what grade are you guys in? Okay, I see grade six. Thank you, Xiao. Anyone else? Six. Xiao Long. Xiao Long, not Xiao. Xiao Long. Yes. Grade eight. Nine, seven. Awesome. So we have a whole range of students. Perfect. So let's get started. So let me introduce you to myself and the team of instructors. So hi, I'm Ms. Haripriya. I am the head instructor for this course. So we are all from Meza Plus. A little bit about me. I graduated from MIT. How many of you have heard of Massachusetts Institute of Technology? You can raise your hand. Put a yes in the chat, right? It's the top engineering university in the world. And I did my bachelor's and master's electrical engineering and computer science in four years. Usually it takes five to six years, but I finished all in four years. I also did a minor in music. My day job is actually, I'm a software engineer at Microsoft Azure, which is the cloud service of Microsoft. And yeah, I love teaching. So I teach a lot of these sorts of AI camps and we'll talk more about other camps we have as well later on in the course. But Mr. Bhagirath, if you'd like to introduce yourself. Yeah. Hi, I'm Mr. Bhagirath. I graduated from Stanford in 2022 with my double bachelor's in computer science and electrical engineering and my master's in AI. My day job is I work at a startup just outside of Stanford, which is actually in the background behind me. And I do all sorts of things from improving networking, working on system and kernel level code, designing hardware, and managing large clusters of computers with expensive GPUs where I'm training large AI models. Awesome. Thank you. You will see Ms. Naomi in the next session. Mr. Richard is here today. Mr. Richard, would you like to introduce yourself? Hi, my name is Richard. Not sure what else I want to introduce. How you know us, how you became a part of the MetaPlus family? So I came to know MetaPlus because I took a camp by MetaPlus about AI a few years ago. And yeah, now I'm working on this with them. Yeah. So Mr. Richard was one of our star students in our AI machine learning research bootcamp. And you'll meet Ms. Minoo and Mr. Coven in later sessions as well. So let's get started. I'm going to talk a little bit, some of the admin stuff before we dive into the AI material. So the syllabus for this course, I think you should be already familiar is we're going to start with intro to AI. We're going to start with decision trees. Then we'll go on some machine learning, regression versus classification, neural networks, con neural networks, GANs and intro to DALI, NLP and chat TPT. Then we'll have a week off for Thanksgiving and then we'll do reinforcement learning. This will be followed by the competition, which I'll talk about in a second. But before I wanted to sort of set some ground rules for the entire camp, make sure that we are courteous and respectful to our fellow students. Please don't spam the chat. Please be nice. Please be respectful and understanding in your communication, right? The instructors are very busy. So if you're reaching out to them, just be nice. And unless it's absolutely personal, please ask your questions in Google Classroom under the questions post. So I can just quickly share what I mean by that. Yeah, I'm going to delete these just because I love that you guys want to communicate with each other. We can create a new channel for that, but I'm just going to keep Google Classroom for sort of my comments. So I can also delete this as well. Hopefully they saw the comment. Okay, so this is the questions channel. So if you have any questions, feel free to tag me. So you can just do tag and then my email address, hpm.metaplustutoring at gmail.com. And just, if you have a question about the course or about the material, please write here. You know, like we mentioned, all of our instructors, we are all incredibly busy because we have our own jobs. That is something as part of MetaPlus, all of us have to be, in addition to teaching students, be pursuing a job or studying AI. And the reason for that is because if we are involved with cutting edge research, we can teach you guys that all the more better. And so we will be introducing some cutting edge research to you, but just be respectful sort of our time. You guys are already math geniuses, right? There's 70 plus of you, there's one of me. So you do the math. It will be difficult for me to respond to everyone individually. So what I would say is just, if you have a question about any of the material, the course just ask here so everyone can see it and everyone can benefit from the answer that I give. And so it's sort of also friendly note there is that students please, you know, write in the Google Classroom. We're more likely to respond to that. We won't be on top of parental emails because you guys are our first priority. So if you have any questions, please ask us directly instead of through your parents, unless of course, you know, you're absolutely sick and you're unable to write to us or anything like that. The reason being is you guys are middle school and high school students. So we wanna make sure that we're also inculcating good skills like communication. And so we want it to come from you rather than your parents. Again, if you haven't joined the Google Classroom link, I already sent it in the chat once, but let me send it again for anyone who has just joined. On Google Classroom is where I will be communicating with you any announcements about homework, about slides. I will be posting homework. I'll be posting lecture slides, everything. Make sure to do your homework on time, right? The homework is a way to engage with the material you learn. And then we will be mostly helping students during the actual class time, right? So make sure to participate, to attend classes, to ask questions via chat if you don't understand anything. Contrary to popular belief, I don't read minds. So if you don't understand something, I won't know unless you ask us. So make sure to ask us in a timely manner. And that's why it's super important that even though all of our lectures are recorded, please attend a class. You don't need to email us if you can't make a class. It's totally okay. We understand. But if you are attending, please do attend class on time. Usually we start our camps five minutes later on MIT time because that's how MIT used to start its classes. But because this is a one hour class, I'd really like to start right from the start of the beginning. So yeah, I think hopefully all of this makes sense. And yeah, make sure that you're on mute unless we ask you to participate, but usually we'll ask you to participate via chat. Okay. Now let's talk a little bit about the competition. December 15th, we will have a proctored competition. It will be 45 minutes long. The rest 15 minutes of the time we'll be spending maybe quickly reviewing answers if we have that kind of time or at least announcing the winners. It will be, if you have taken a Math Kangaroo exam, it will be structured similarly. There'll be three point questions, four point and five point questions. There will be, unlike Math Kangaroo, in addition to multiple choice, we will also have some free response questions, meaning that you will have to write the mathematical answer. Unlike Math Kangaroo also calculator and handwritten notes are allowed. So this is very similar to an MIT final exam that we were always allowed to use our notes, but you can't ask from outside help. So no asking parents, grandparents, siblings, et cetera, no asking to FGPT or any other AI or other resources such as the internet or the book. So it's all your notes, but you do have access to your notes. So make sure to take copious notes. I will give you a pro tip and I'm gonna give this only once. At MIT, what we used to be allowed was we would have a cheat sheet. And so a cheat sheet can only be one page front and back. And everything that we had learned in a course, we would write just on that one cheat sheet. And that is actually very useful because when you start writing your own cheat sheet and trying to think about what's the most important things that you need to know for the exam, that's gonna be very helpful because as you are writing it, you're also memorizing it. So what used to happen is I would make a cheat sheet, but then I would realize at the time of the exam, I didn't even need it because I'd already memorized everything just by writing it. So I will give you that as sort of my one suggestion instead of having millions of pages of notes, probably having a cheat sheet would be better, one page cheat sheet, but again, it's up to you. We don't believe in out of syllabus content, right? This competition is based on stuff we have covered in this class. So I promise you every question on the competition will be something we have covered in class, but it will be difficult because we might combine concepts from different classes, right? So I would say my biggest suggestion to all of you is don't get cocky. I know some of these concepts might be easy to start with, but trust me, all the MetaPlus instructors have a knack of asking hard questions. So make sure to constantly review your class lectures and homework. And sort of the final thing I wanna say is, if you're not understanding stuff, make sure you're asking questions, but don't worry about it, right? This is not a competition. I know this sounds so cheesy, but this is not a competition between all of us, right? It's a competition with yourself, sort of. You wanna improve yourself and kind of you're on your own learning journey. So don't worry about it if you feel that other people are understanding things quicker, or it doesn't matter, right? We're all, first of all, we're different grades. We have different mathematical levels. We have maybe different exposures to AI. So we're all in our different space. But again, if you need any help, we're willing to help you. None of the instructors are going to judge you. We wanna make sure that you learn this material and you learn to love it. So with that, we'll start with what is AI? This is an interactive course. So what I'm gonna do is I'm gonna launch a poll here of what is AI. And I want you to sort of answer in your own words, what you think, like what are your initial assumptions and stuff like what is AI? Yeah, write anything you want. And I wanna end the poll in one minute exactly. And hopefully I can see all these answers. We'll see. I've actually never launched a short answer poll before I've only done multiple choice questions. So I'm curious how it's gonna show up here. 30 more seconds. And I'm gonna do a abrupt close at one minute just because there's a lot of material we wanna cover. 15 more seconds. 10 seconds. Five, four, three, two, one. Sorry, I see just a few more people are like finishing submitting. So I'll give you the extra few seconds. I don't wanna interrupt you mid-thought. Okay, I'm gonna actually close it now. If you didn't get to finish, that's fine. Let's see. Oh, okay. Some people, I'm not sure if you guys can see the answers, but I can just read it all. A lot of people, you just defined what is AI. AI is artificial intelligence. It's a platform that pulls information for various sources to answer a question. Interesting. Something similar to chat GPT. That's part of it, but we'll talk about it a little bit more. Xiaolong, if you wanna ask your question in the chat, feel free to ask it. Artificial intelligence, computer intelligence. Yeah, it uses algorithms. That's awesome vocab word to proctor the most detailed response to the user. Awesome. Intelligence like chat GPT. So a lot of you guys are familiar with chat GPT. That's interesting. It's like human intelligence, but it's complicated. It acts like a human brain. Think like it. These are great answers. So you guys have a definitely nice basis. So probability-based answers, yeah, I like that. A lot of probability, a lot of math. It uses pattern to make decisions. That's key. Different algorithms, right? So now let's talk about what is AI. So all of you guys were totally on the dot. ChatGPD is just one example, but there are lots of different AIs that we're going to talk about here. So what's AI? AI is artificial intelligence, and it's the artificial intelligence demonstrated by machines. And so what these machines are trying to do is they're trying to mimic humans in their learning, their reasoning, and their problem solving. AI can be rule-based or learning-based. Today, we're going to talk about rule-based AI. Most of the other sessions are going to be focused more on learning-based. Rule-based AI is fairly easy. It's just like, hey, if this is the condition, then this should happen. If this is the condition, then that should happen. Similar to if-then statements and things like that, today, like I said, is rule-based. And so you will see that that's kind of easier, quote-unquote, than learning-based, where we won't really understand what's going on as much. So AI can include many fields, such as machine learning. So that's learning patterns from data. It's a subset of AI, like all of you mentioned. We can use, in this class, AI and machine learning interchangeably. Natural language processing is basically AI relating to words, relating to human language. It can be any language. Then you have computer vision. Sorry, let me go back here. You have computer vision, which is all about AI and images. So we have a question here in the chat. Is AI basically a function that can define the questions that will be asked? So not quite. I would more like say that it tries to understand patterns from the data you give it. And then, yes, it can answer whatever you are trying to find out, sort of. So yeah, you're kind of close. So where can AI be used? So ChatGPT, all of you. How many of you, out of curiosity, and you can just do show of hands, use ChatGPT for your homework assignments? I promise I will not tell your teachers. Oh, we have a bunch of really good students not using ChatGPT. I'm not sure if I should believe that, but I will assume you are all honest group of students. So AI can be used in health care. It can be used to diagnose diseases, right? So for example, you show a bunch of images, maybe scans of something, like x-rays, chest x-rays. And does a person have a disease or not? So instead of having a doctor analyze the image, you input a bunch of these kinds of examples of x-ray images and whether there is some disease or not. You input it to AI. It starts understanding patterns from the images. So when you give it a new image, it's going to totally understand, oh, yeah, this person has a disease. So it can be used for other things as well, like predicting patient outcomes and personalized treatment as well. So what's going to happen to the patient if we administer this medicine? I had a question about CHAT-GPT, that if you ask it the same question twice, both answers will be different but have the same general idea. Yeah, and it's because of the probabilistic sort of nature of machine learning that it might give you different answers. But we'll wait about CHAT-GPT. We'll talk about it more in our natural language processing unit. So we definitely have a lecture dedicated to CHAT-GPT and how it works. So other applications of AI, finance, fraud detection. So how do you know if you're looking at spending history of a bunch of customers, how do you know that one person out of there is a fraud? So you try to, again, find the anomalies, find the patterns. Find the patterns. I keep on saying that. And try to see, OK, people who have this kind of spending habit, they likely are frauds. Credit scoring, automated trading strategies, algorithmic trading, all of these sorts of predictive nature dealing with stocks and things like that, all of that can use AI. It can be used in retail, which is just a fancy word of saying shopping. So it can give you personalized recommendations. If you bought maybe different shades of pink t-shirts, maybe it will start recommending you pink pants or something. I don't know. Maybe that was a silly example, but something like that. Dynamic pricing, so maybe depending on the time of the day or something, items on the internet can be priced differently. And so they might price it as per customer's spending habits. Inventory management, how many things do we have in our shop, just having a predictive tool because we might not know exactly the number of items we have in stock. Automative, if you have ever seen an autonomous vehicle, which means like a driverless vehicle where it doesn't have a driver, then that's an application of AI. Like, how does the car see what's out there and how does it understand? So those are all applications of AI. OK, so I'm going to just finish this up and then I have a question. So marketing is also a great thing. It tries to understand what human beings are looking at, what else should be recommended to them. It analyzes the sentiment, like are humans happy to see something, not happy to see something. Customer segmentation, so like if you can group a certain type of customers and you can say, oh, all these types of customers are fairly similar. So if customer A and customer B are similar and if customer A buys something, you can recommend the same thing to customer B. So that's sort of like the different applications of AI. And feel free to, now that you are in an AI class, take a look at the news, make sure that you are in tune with the different applications of AI that you see. Question, are we going to talk about how AI comprehends the data as humans don't know how AI comprehends? Oh, give us examples of the work. So yeah, we will definitely talk a little bit about how AI interprets this data. In fact, each of our classes is going to be dedicated to that. So I heard the word algorithms, right? Someone answered that in their poll question. So algorithm is basically some sort of set of rules to follow for the computer. And so we have many different algorithms for AI. And we're going to actually learn all of them. Supervised learning algorithm is sort of the first one. That's when you feed in the computer lots of data and labels. And so, for example, if you're trying to identify if a given picture is a picture of a cat or not, what you're going to do is you're going to feed in the computer hundreds and thousands of images of cats and dogs. And you're going to say, hey, this is an image of a cat. And you're going to literally write the label, which is cat. So the image is the data. And then the label is the actual, like, this is a cat or this is a dog. And so you feed in both of those things, and hundreds and thousands of them. The computer finds the patterns. And then when you show it an image that it hasn't seen before, it can say, oh, yeah, this is a picture of a dog. So that's supervised learning. It has the data and the labels. Unsupervised learning is interesting because you don't have the label. So you don't have the actual label of, hey, this is a cat or this is a dog. And so that might be a little bit confusing to you, like, how can we do this when we don't have labels? But we'll talk more about that in future weeks. And then the last one I wanted to talk about was reinforcement learning. And that involves states, actions, and reward. So this is a little bit different from supervised and unsupervised. And it's more about machine learning in games and those types of situations. I am seeing some people are writing notes. Lovely. That's fantastic. I will also be sharing the slides later. If you feel like you're rushed or something like that, don't worry about it. And in fact, this slide is going to pretty much show up every week for us because this is very important. So one of the vocab words for today is interpretability. And I think I just got that question. How do we know how AI can comprehend the data? And so that comes under interpretability. What does it mean? Interpretability is the, yeah, you get the recording of the session along with the slides. Interpretability is the degree to which a human can understand the cause of a decision made by AI system. Now, when we do rule-based AI like how we're going to do today, you will easily understand why the AI is making a certain decision. As we get later on in this course with more and more complex models, we are not going to have that level of interpretability, which means we're not going to really understand how AI is making those decisions. And that can be a bit of a problem, as you can imagine. So it's really important to understand the reasons for the decisions because decisions can have significant consequences. Think about like an autonomous vehicle. You want to make sure, how is this AI model deciding that there is a person walking on the street and we shouldn't run the person over? And so more transparent models allow for better identification of biases, which just means basically understanding what sorts of decision the model is making and why are they making it. Are they kind of trained to do a certain thing which is not good? And building of trust. We want to trust the AI model. Visualizing a model and its representations can be helpful in improving interpretability as it allows for insight into what features influence the model's decision. So we want to, again, know why the model is making the decisions it's making. And so if we can visualize it somehow, it's a lot better. Think about if any of you have worked in three dimensions, like in math class, it's kind of hard to visualize. But if you maybe work in two dimensions, you can easily draw diagrams and you exactly know what's going on. And sort of that same idea that if we can visualize it somehow, for us as humans, it's so much easier to understand what's going on. We'll talk about unsupervised learning. I know I had a question about it, but I'm not going to respond to just quite yet because we definitely will talk about it in later sessions. And I don't want to confuse you all just this minute. But today's topic, one of the AI models we're going to talk about, right? And we talked about so many different AI applications and they all use sort of different AI models. And each AI model has its strengths and weaknesses and has its particular applications. Today, we're going to talk about a very simple one, which I'm sure you have seen before, decision trees. How many of you have seen, and just look at the diagram, you might not have heard the word decision trees before. How many of you have seen this type of diagram, some sort of tree-like structure? You can just raise your hand. Yeah, a lot of you, right? You probably didn't even know it, but you were kind of doing something that AI is doing. So what is a decision tree? Well, it's an algorithmic for predictive modeling. So it's a supervised learning algorithm, right? Supervised learning algorithm means what? What did I say? Does it have data? Does it have labels? Does it have both? Does it have neither? Does it have one? Write it in the chat very quickly. Data, labels. Yeah, cool. I'm seeing some correct answers. And I'm not gonna right now say what the answer is, because there's a question coming up soon on the poll. By the way, the poll questions that I ask, it's not being graded. It's just kind of to keep it interactive, keep it light. So the idea here is decisions are structured based on input data, right? The decision tree on the right, it's about scholarship eligibility. And so each of these are statements of, they're kind of true or false statements, right? Is the individual's, is the student's GPA greater than 4.5? If it is, then we go and take the left branch. If it isn't, we go and take the right branch. So if it is greater than 4.5, then we check whether they participate in extracurriculars. If they do, then they are eligible for a scholarship. If they're not, they're not eligible for a scholarship. If they don't participate in extracurriculars, they won't be eligible for a scholarship. If their GPA is less than 4.5, they're not eligible in a scholarship, right? So here is my second poll question. Oops. That is exactly why I have a fake answer page, because I knew I would do something like that. So let's launch this. Again, you'll have a minute or 30 seconds if people are faster. What type of AI algorithm is in this entry? Check all that apply. So there can be one answer, there can be multiple answers. It is an unsupervised machine learning algorithm. It's a supervised machine learning algorithm. It's an algorithm that involves data and labels, and it's an algorithm that involves data only. And this one I honestly will not extend it just because I want to keep on moving here with the slides. Five more seconds, three, two, one, closing the poll. Cool, so a lot of you said B and C, and that is indeed correct. Decision tree, we just covered it in the last slide, that it's a supervised machine learning algorithm. And supervised machine learning algorithm, as we talked about it previously, involves data and labels. So let's keep that in mind. And now I will turn it over to Richard. So let me give you remote control, Richard. Give me a second here. Okay, you should have control. Hi, yeah, so next we're going to go over some vocabulary related to decision trees. So you know that decision trees get their name from their tree-like structure. It has a root, branches, and leaves. The root node is the top node that represents the original choice or feature. It is the starting point of the entire tree. So on the right, you saw this example earlier, but the root node starts by checking if your GPA is greater than 4.5 or not. Internal nodes, or decision nodes, are points where the data is split based on a feature. These nodes test an attribute to divide the tree. In our example, the decision node checks whether or not you participate in extracurriculars and splits the tree based on if you do or don't. Leaf or terminal nodes are the endpoints of a decision tree. They represent the final outcome of classification. In our example, they're the outcomes of if you're eligible for a scholarship or not. Branches, also referred to as edges, are the paths that connect one node to another. They're based on a decision rule and represent the flow from one decision to another in the tree. Splitting is the process of dividing a node into subnodes based on a decision criterion. This is how the tree grows and how different conditions are explored to reach a final one. A parent node is the node from which a split originates. It is any node that has other nodes coming from it. These include all of the nodes except for the leaf nodes. Child nodes are nodes that are created as a result of a split from a parent node. Each parent node will have at least two child nodes when a split occurs. All nodes except for the root node are child nodes. A decision criterion is the condition used to determine how data should be split at each node. In our example, they're the questions of if your GPA is greater than 4.5, and if you participate in extracurriculars or not. All right, with this, let's go over a simple example. In our new example, a decision tree is used to check if a person is eligible for a loan. It uses features of income and credit score to make the decision. To construct the tree, data is split based on the conditions of the features until the final decision of eligibility is decided. In the root node, we check if income is greater than $100. If not, then the result is not eligible. If it is, we check if credit is greater than 10. If that's also true, then the outcome is illegible. Otherwise, it's not. This example is pretty unrealistic, but let's use this example to do a question to check your understanding. This is the question. If an individual makes $110 and has a credit of 8, will they be eligible for the loan? Please answer yes or no. I think there should be a poll that you can answer in. Yeah, let me, I will kick off that poll. Oops, I'm sorry. Let me kick off that poll. Oh, sorry. I'm going to just stop remote control for a second, Richard, just so I can launch this poll. Oh, poll is being shared. That's why I can't launch. Where is the poll being shared? I'm kind of confused here. Stop sharing. Okay, sorry, launch. Okay, yeah I'll give you 30 seconds since I spent a lot of time trying to find out how to launch the poll. Three, two, one, zero. I'm gonna share results and I'm gonna stop sharing so I can launch the next poll. But yeah, sorry, Richard, you can continue. So yeah, it seems like most of you did answer no for the question. And that's the right answer. Thank you. So yeah, the right answer was no because although income is greater than 100, credit is less than 10. So the answer is not eligible. All right, then where are decision trees used? Decision trees are used in many different classification and regression tasks. They can be used with many different kinds of data sets. Some real world applications include medical diagnosis, credit scoring, marketing, fraud detection, and a lot of others. All right, then why would you use a decision tree over other complex models? Yes, there should be a poll. Just think about this question. Yeah, and I just launched a poll. I'll give you guys a minute. Just jot down some ideas. Why do you think you'd wanna use a decision tree? And we haven't talked about more complex models. So it's totally okay. It's more of a, Okay, we're over a minute, so I'll give you maybe 15 more seconds. Okay, and I'm gonna end the poll here. Let's see results. um so yeah they're more understandable straightforward for people to interpret so we use the word you know interpretability uh it's more uh efficient maybe uh if since it's like sort of simple uh you can understand right so that yeah anything with and we have the word interpretability anything with interpretability understanding it exactly but feel free to continue um all right yeah so most of you in your answers did list like some of the pros of a decision tree and these are yeah some of the pros and cons of decision trees advantages of them include being easy to understand and interpret where the tree-like structure makes it easy to visualize and explain the decision-making process they can handle both numerical and categorical data that means you might not need to pre-process the data as much they can automatically identify important features when training they automatically identify which features are most important for making decisions they can also handle missing values and outliers without significant impact that means they can still perform well even if the data isn't perfectly clean decision trees however do have disadvantages such as being sensitive to small changes in data where slight variations can cause a completely different tree structure to be formed they might be limited in generalizing if the training data isn't representative so you want to avoid overfitting on the training data to make sure that the model can work well with other kinds of data they also might be biased if the data is imbalanced this could result in the model favoring one class over another leading to inaccurate predictions yeah let's turn it back to Sripya awesome um okay so now comes the fun part the mathematics right uh that i'm sure you're all waiting for uh so we're going to talk about how do you make these decision trees right because obviously you have some small data set you can figure out and make the decision tree by yourself but when we are talking about using ai for something that means you have hundreds and thousands of examples you have very complex decision trees that need to be created and now you need to have a method method you need a method you know what i'm saying you need to have a nice methodology there uh to um you know create the decision tree there has to be some reason um some method to the madness if you will um and so that method to the madness is math of course um so all of ai you'll see it's not magic it's math um so how do we create this decision tree so let me first go over some vocab words and formulas um it won't make sense uh right off the bat but i'm gonna kind of in the last 15 ish minutes of this class uh walk you through an example and your homework will be similar to this example so that way you will understand what's going on how these decision trees are made automatically by the computer by the ai essentially so the first word here is a gini index or gini impurity uh it's a metric to measure the impurity of a data set by calculating the likelihood a randomly picked instance is misclassified uh and now i'm just forgetting is it gini index or guinea index uh mr big ears or mr richard totally bad with pronunciation mini index guinea right guinea yeah oh genie yeah genie uh sorry i i'm the worst person with pronunciations but genie index uh it's a metric to measure the impurity of the data set by calculating the likelihood a randomly picked instance is misclassified so impurity essentially um we will see that but it's like it's basically how much of this is being misclassified how much of its sort of classes are are different um and it'll make a little more sense when we do our example and so this is our formula so has anyone seen that like one minus anyone seen that uh funny symbol before is it new to anyone raise your hand if it's new new yeah new to a lot of you so that's just a uh it's a sigma so it's it represents summation summation just means we're going to add up all the stuff and i'll give you a quick example uh when i'm on my ipad uh but basically the p here stands for the proportion of elements um and then the class i and the node is n which is the number of classes so um n is the number of classes so uh we're gonna i'm gonna show you exactly what this summation means it's a scary looking symbol but it's it's stuff you already know and then if you want to do a genie impurity for a split uh or weighted genie impurity then um it looks something like this where the w is the weight and then you use the genie index and again you sum all of this up from node equals one to the nth node okay so here's our example question uh you are given a small data set related to loan approval system for a bank the bank contains the data set contains information about the customer's annual income credit score and whether the loan application was approved or denied the data is classified into two categories whether they're approved or denied right so first of all i want you to calculate the genie index for the root node before any split then once we start splitting the data i want to try to see if we split it on annual income what is our new genie index and then see if it reduces the impurity of the data set and all answers should be rounded to three three decimal places and this is what our data set looks like obviously i'm giving you a very simple data set which has one two three four five six seven eight rows in the real world you would have hundreds and thousands of roles you would have so many more features and that's why you need ai right why do you need ai can a human do all the things that you know we have talked about of like all the different ai applications 100 but a human is going to be much slower ai because it's a computer right it can do everything much faster and so um our first question is uh let's go back here i don't want to show you the answers quite yet so let me uh stop screen sharing from here and let me i'm going to share my ipad um and i have already copied down let me see how i can share i've already copied down all the information we need okay hopefully you guys can see it hopefully it's not too tiny i copied down the formulas and i copied down um i copied down the data so what i'm asking you is first uh let's calculate the gini index for the root node before any split so we're not splitting anything so basically i'm asking you just look at the data okay and we have to see first of all how many of our approved so we're just looking at the final column here because this is before we are making any splits in our decision tree so we look at this uh and i have copied this over uh several times we look at this column and here let me copy this actually even once more just in case i need it again so this is our data we want to make a decision tree based on it um okay so since we're not splitting yet on either of our features and these are our two features right annual income credit score we'll talk more about features in later workshops but basically features are anything that you are trying to use to predict something and so in this case we're trying to predict loan approval from annual income and credit score and so uh we aren't using any of those features yet it's the beginning of our decision tree so we just had to look at our loan approval column and so the first thing you're going to do is you're going to count how many of them have been approved and how many of them have been denied and how many rows there are in total so feel free to just uh type it in the chat how many of the uh how many approvals do we have here let me actually just write it in a font that we can see color we can see how many rows right three i'm seeing a lot of threes how many denies five perfect so we have a total of eight rows okay now before we calculate so we want to calculate um the gini index here we know that this is the formula for our gini index right before i calculated i want to just show you what the summation sign means so if i say something like this and this is unrelated to the uh to the given example i just want to kind of teach you a little bit about summation so if i say i equals one to i equals five and i have this fancy sigma symbol and i have i square what i do is i just do plug and what i do is i just do plug and chug and i start with one so we how to plug and chug into this so we say one square now summation i said it's nothing it's it's a sum here right summation is sum so we're adding stuff so then after one we have to go from i equals one to i equals five so we'll say plus two square plus again we just replace the i uh plus four square plus five square do we add a plus six square yes or no just uh i'm gonna see whose videos are on yeah i'm i'm seeing no's right we don't add a six square because we are just going from i equals one to i equals five um and so that's the last here um number and we're just plugging and chugging into this i square if i gave you something like this i equals one to i equals three and then i would say something like i then if you can quickly write don't give me the final answer but how would i write out the next step if you can quickly write it in the chat yeah one plus two plus three we just plug in chug and we just go from i equals one to i equals three so hopefully that makes sense uh oh let me erase this uh oh shoot we have nine minutes so i'm gonna rush through this last example here um but what we're going to do is we're going to plug and chug into this equation so to calculate the gini index what we're going to do is it equals to one minus and then i'm going to do the summation of the different proportions so how do we do that well let me put a whole massive parentheses because i want to sum up everything in here and the proportion is nothing but approved over total so that's three over eight oops three over eight square plus denied over total because those are our two options okay does that make sense so far we take the proportion of each uh approved and denied so there are three approved eight in total so three over eight five denied eight in total so five or eight but since it's pi squared it's the proportion squared we're gonna say three by eight bracket square plus five by eight bracket square and we subtract does this all make sense give me a thumbs up thumbs down if you're like i'm totally lost thumbs middle okay uh so one more time we're we're plugging and chugging yeah why do we have a one minus that that's a good question so that's how gini index is defined that's the formula one minus this okay so we're just plugging and chugging into this formula i can write this formula a little bigger i'm gonna kind of delete the fancy symbols here just to kind of not confuse you all but one minus the summation of the different proportions and so the different proportions are three over eight and five over eight bracket square we sum it all up and one minus that are we all good now kind of uh just trust the process i know you might be like why why am i doing this so the gini index for this is 0.469 okay i'm gonna uh i have a couple of poll questions but i think i'm just gonna talk uh talk you through it rather than asking just because we have uh we're kind of short on time so the question that i have is what's the ideal gini impurity what do you think is the ideal impurity is it uh 0 0.5 or 1 okay i see some answers in the chat feel free to answer in the chat we're going to make this a chat question okay i have answers all over the place and that's fine because we still don't really understand what impurity is so the impurity the good impurity is zero uh so that means because right now basically what i'm saying is if i'm making if i'm going to make a branch right or something you have not all of these are don't have the same classification right you have some that are approved some that are denied and so the idea is we want our impurity to be zero right when you are doing a decision tree when you're making a decision tree uh chart you want to make sure that everything that falls under one of these like leaf nodes when we talk about anything that falls under they all are accurate like the classification is accurate so the impurity that we are always trying to aim for is zero yeah so what's the range of gini impurity so i because i i'm getting these questions in the chat you guys are like one step ahead of me which is awesome what do you think is the range of gini impurity does it go to zero to 0.5 0.5 to one or uh zero to one ah i have tricked you so all of you are saying zero to one but this is a trick question zero part you're correct obviously but uh so let's just do one more page here the zero part you're correct uh because that means we have if we have correctly classified everything then our impurity is zero but what is the worst case scenario of uh incorrect classifications can someone write in the chat what is the worst case scenario of incorrect classifications uh not the number but like what what do i mean by that correctly classified means you know everything is in the proper sort of if you go if you use that data row on the decision tree you will classify correctly everything is wrongly sorted so maybe like uh i guess incorrect classification maybe not the best way to put it like the worst case scenario is when you have a like a branch and half of them are kind of uh half of them are approved and half of them are denied that's sort of the worst case scenario right uh i guess that's incorrect classification sorry that was not the best way of putting it does that sort of make sense or are we totally confused now okay um we have four more minutes can you guys stick around by the way for this uh lesson okay cool um so i won't rush then and you can catch up with the video okay uh let's let's put it this way um we have maybe some trees like this some sort of decision tree like that right and maybe if if you remember the decision trees that richard and i had gone over before uh you know that uh basically it was yes or no yes or no right uh and it's very easily categorized categorizable but what if i gave you a data set row where if you actually go through this branch then this should be this would come out to be a yes right then that's an incorrect classification so my claim here to you is what if uh when we go through our entire data set uh we have uh some incorrect classifications um in that uh maybe in this branch we will end up with uh like we have five denies and three approves or something like that then that's not a we're not done with our decision tree right because we want to make sure that all of them are denied or all of them are approved does that sort of make sense or so confused if you're confused fine i can explain different way if someone can write in the chat are you confused confused okay let's let's let me let me make this less complicated let me start from the beginning here okay i know i know a good way to explain this let's start from the beginning right now we have uh three approved and five denied right three approved and five denied can we have uh can we have a decision uh can we say that we already have like a decision tree we can't right because when we make a decision tree we all of these aren't the same classification right so you need to make some sort of decision tree are you with me so far you had to make some sort of decision tree so that you have some sort of like demarcation the worst case scenario if the best case scenario is like if all eight of these were approved then you're like okay i don't need a decision tree i'm done and so that's a gini impurity of zero with me so far but the worst case scenario is that if eight were denied what's our gini impurity zero i'm seeing like interpretive dancing love it uh zero so then what's our worst case scenario that's still not our worst case scenario what's our worst case scenario of approves and denies how many approves and denies would we have for it to be a worst case scenario yeah exactly four and four awesome you guys are getting it love it you had to be four approved half and half yeah exactly that's that's what i'm trying to say here and this is this doesn't only go for the root node this also goes for other other branches as well right and we'll talk about that a little bit more but for now let's just stick to the root node here so four approved and four denied so now if we put it into our equation here what is our answer one minus what's the proportion here you have four approved four denied what are the two different proportions four by eight right or a half four by eight four by eight right so that's like what half squared one minus two one over four so the answer is half correct okay cool uh if you had to drop out by now feel free to just catch up with the video to 0.5 that is our um range of gene impurity now what do we all does it all make sense why it's not zero to one it's zero to 0.5 yeah okay cool so very quickly let's go on to the sample question oh why is it one uh sorry why is it why is it one half uh because four approved and four denied is our worst case um how do you get a one impurity if the worst case in here oh there is no uh one impurity that's what i'm saying that is zero to 0.5 that's the 0.5 is the maximum uh gene impurity you get for a two class uh problem does that make sense lucas does that make sense or still confused there can be anomalies to the data row and that's okay and we're not going to talk about that today at some point you had to stop the decision tree why is it one minus one half yeah so i just i just fit it into this equation one minus summation of p square equals one minus four by eight square plus four by eight square and so then i just i just substituted in this equation does that make sense okay cool you know why is there one one half and one one four oh i just uh the four by eight square is just four by eight goes to one half so that's one half square right if i want to write this out it would be one minus half square plus half square equals one minus two one by two square does that make sense sort of lucas or still confused OK, cool. Do we all get that? So here's what I'm going to do. I'm going to change up the homework assignment a little bit and I'm going to change. I'm not going to kind of teach the rest of the material because it gets a little more complicated. Yeah, you can do whatever math as long as you're following the things. I'm just trying to write out all the different steps. So here's what I'm going to say is. Essentially, what we are trying to do here. Is we are trying to and we can we can finish actually part of this lecture next week. It's fine. What we're trying to do here is we're going to. We're calculating the Gini impurity. Right. So the Gini impurity is telling us that we need to that we need to create a decision tree for this root node. If let's say this was not the first row, like it was not the root node of the decision tree and it was somewhere below in the decision tree. Right. Somewhere here. And if we had that, three of them are approved and five of them are denied. Or let's do a different maybe number. We would say maybe one of these leaf nodes is actually right now giving us one approved and two denied. What do you think we need to do? Are we done with our decision tree or do we need to keep on creating one? Keep on like adding branches or whatever to it. If we had enough features. Yeah, we keep going. Now, in theory, you do stop at some point and sometimes you will get some misclassifications. That is life. But for now, we're going to say, yeah, we'll just keep on going until we get 100 percent correct classifications. So I'm going to stop right there, actually, and we'll continue this lesson next week. But what I want to kind of drill down the point is how do we get the the Gini impurity? Do we understand the concept of Gini impurity and like why we're kind of doing it? Gini impurity sort of tells us how far you need to go with the decision tree. And the idea is you want to kind of decrease that impurity. You want to make sure it's all correct classifications. Does that make sense? Any questions? Any last questions on any of the things we have talked about? We'll continue a little bit more about Gini index next week. About how we do it in the future branches, but I'm going to give you a short homework assignment. I'm going to modify it based on the content we covered today, so I can actually share my screen right now. Yeah. OK, good question. When would you use Gini index for AI? The AI is using Gini index to decide how to choose these features. And so we haven't gone to the second part of this, which we'll go over next week of how do you choose, like whether annual income or credit score should be the next feature that you are branching off on. Right. Yeah. When is the homework due? That's a great question. Homework will always be due with the beginning of class next for the following week. Gini impurity is calculated for each node, right? Exactly, Rowan. And so we're going to talk about how we do it for each node next week. But for now, I kind of want you to understand how Gini impurity works for just a single node and just for the root node. Right now, we're only focusing on root node. So I'll give you a couple of homework questions like that. Is there a certain formula? Yeah, we're going to use the splitting formula that we talked about. But again, we'll talk about that next week. Any other last question? Yeah, I will post the homework on Google Classroom. And the formula doesn't change for any problem. It's basically the formula saying you just need to kind of understand how to do it. Any other questions? Oh, code to the Google Classroom. Yeah. Let me send that to everyone. And if anyone has any questions in the meanwhile. Okay. Any other last-minute questions before next week? We'll continue this topic so I'll make sure that I'll ask you questions that we only covered today. Yeah, if you can't do homework due to school projects, like I completely understand. We're not gonna grade each one individually, it's more for your own edification. We will do take a quick look just to make sure you're understanding the material or if we need to kind of review, revise something for next week. Any other questions? Yeah, no, no, not graded for the homework. The only thing that will really matter is the competition at the end, so we want to make sure you're prepared for that. So again, if you have any questions, make sure to ask them. What do we do to fix the impurity? Yeah, we'll talk about that next week. That's a great question. Anything, anyone else? Otherwise we'll close the video. Okay, well, thank you guys. Great job participating and I'll also post the homework and I'll see you guys next week and we'll continue to talk about impurity. Yeah, the equation does solve impurity, sort of, the equation in the decision tree. Bye!

artificial intelligence

AI applications

decision trees

supervised learning

Gini impurity

healthcare

fraud detection

autonomous vehicles

MetaPlus

interactive session