OK, so we're just going to be starting to go over the first question of the homework. We were asked to use apply average pooling on this feature map. And let's go to the key and see how we could do this. So if we're doing average pooling, we would take the average of each of these four squares. So each value divided by the number of values there are. So there's four in each, so we divide by four. We continue to do this for all of the four squares. And this would be our new value, which everything is average pooled by. Now moving on to the dot product of the image and filter. So this is our image. This will be our filter. And remember, we just apply where the filter kind of goes over the image, and we just shift it. And then we can apply the dot product. So here again, our image and filter. So applying the dot product, we multiply 4 by 1, negative 1 by 0, negative 3 by 2, and we add this all up. So our first value is negative 2 in our new value. And then as you can see, we just multiply whatever lines up. And then we shift our kernel over. So we continue to do this. Our next value we get is 3. Remember, adding everything up, multiplying it, and then adding it together. And then we continue to do this one more time. And then our last value is negative 3. So this is our answer. Now, I'm just going to share this link with you guys in the chat. You can, by doing this, you can upload. You can connect the Teachable machine with your Google Drive. If you have, so you can save any projects you guys did. And does anyone have any questions? Naomi, if you want, since I have it on my Google Drive, I could possibly share it. I think I can. I don't think we can hear you, sorry. I can hear. Yeah. Oh, you. Me? I can hear, too. Yeah, I can hear. OK. It's just a skill issue. Let me see if I can log in. Give me a second here. I can show you, just in case you didn't get to this part. Can you give me host permissions Naomi? Yeah, sure. So, in case you didn't get to this part, or you're just not entirely clear on how this works, kind of for homework, I just wanted you to see how the training process works for machine learning, and especially for an image recognition. So this isn't the exact same thing you did for homework. I think for you, it was like chihuahua and muffins. But I did it for 100 cats and 100 dogs. And then I just pressed train model. And what it does is, let me see if it takes a second, it'll show like 1 over 52 or 53 over 50, so on and so forth. And so what that is, is different iterations. So like, yeah, you can see the zero over 50. Hopefully, I'm not sure if it will work since my Wi Fi might not be the best right now. Okay, there you go. Each of these numbers that you're seeing, it's like, oh, this is going over the all the images once, twice, thrice, fourth, 50 times over the same set of 100 images of cats and dogs. And you can always export your model and use it and stuff for other things. But let me not use webcam, let me use file. And if you choose an image of a cat or dog or, you know, something, I don't know, okay, there you go. I don't know if I have any images right from the top of my head. But let's see, yeah, I don't even know what this is image of, I'm on my brother's computer, but it's going to predict something. And so what's interesting about this image that I uploaded, you can see it's clearly not a cat or a dog. But because the model has been only trained on these two classes, cat and dog, it's only going to show up as one of them and you'll see a percentage. It's kind of unfortunate for this, it's like saying 99% likely that it's a dog, but that's what it is. So it can't predict anything that's not one of the classes you have trained on. You can always train on more classes. But the way to read this output, because I saw some of you write it in Google Classroom, and I'm just going to modify the language that some of you may have used a little bit. The way to read this is that this image is 99% likely to be a dog and 1% likely to be a cat. So what you're going to say is whatever you have the higher percentage of, even if it's like 51, 49%, whatever the highest percentage is, you take that. So in this case, obviously, it's dog and you say the classification of image is a dog. So that's how you would kind of say it out loud or write it down in a report or something. But yeah, that's all I had. Great. Does anyone see? I can go back and explain the second problem of the homework again really quick before we get to the slides. So remember, we're just taking the dot product of the image. So remember, this is our image and this is our filter. So we multiply the values that line up with our filter in our image, and then we add them all up. This is the first value of our new image. And then we just continue doing this as we shift the filter down. So our first value would be negative 2. And then here we see we shift the filter, and then we multiply the values that line up, and then we add them together, and then we continue this process. Does that make sense? Okay. I think we're all good. I'm going to quickly start my share in a few seconds on our slides for today. Perfect, so today's lesson is on generative art, but we're also going to talk about random decision, random forest model and decision trees. So first, just a quick review on standard deviation. So standard deviation measures the amount of variation of the values as a variable around the mean. So here we have our bell curve, this value here, this is called mu. It means the population mean, as you can see right here. And then each of these lines is the mu or our mean plus our standard deviation. And then each line is another, just plus our two times our standard deviation plus three times our standard deviation and so on. So N is our population size. So the way we find our standard deviation is we take the value of each, we take the number of each value of our population subtracted by our mean. We square that and then we divide it by our population size and we finally take the square root of that. Okay, so now we're going to use what we just learned and apply it for an example. So calculate the standard deviation of the number of leaves Dr. Rue ate during the week. And this character is Dr. Rue. So we have our data set and these are the number of leaves Dr. Rue ate. So we would calculate this by this equation at the bottom. So we take, remember, every value we have subtracted, I'm sorry, by the mean. And then we take the square divided by the total number of data points we have and then we take the square root of that. And this is our standard deviation. Okay, so now we're going to talk about decision trees. So how decision trees work for regression or they can work for regression or classification. And given the decision node and decision nodes are just the nodes that have sort of decision on them, like bedrooms being less than three or has a backyard, it wouldn't be the, oops, sorry, the values. These would be called our leaf nodes, but given the decision node, the data that best fits the condition would continue to that corresponding node. And for this example, we have a regression model. So our bedrooms being less than three. So we have less than three bedrooms, yes. We have a backyard, yes. Then our house we've estimated to be $60,000. Now, how do we get the values of the leaf nodes? These are just the averages or means of the leaves of the decision tree. So if we're going to take bedrooms being less than three, no to that, so our bedrooms would be three or more. Then let's look at all the data sets or data points that have this. So this wouldn't, first one wouldn't be. Second one, yes, we have four bedrooms, so that's $140,000. This value, yes, we have five bedrooms, $160,000. We take the mean of this, so 140,000 and 160,000, add them up, divide it by two, $150,000 would be our mean. So we are given the standard deviation of this data set, roughly $57,000. It's a pretty big difference between the values. So let's take two features and calculate the standard deviation of house price for both of these features. So remember, our two features are our bedrooms being less than three and the house having a backyard. So this is how we would take the standard deviation of a feature, and we're just going to apply this to having a backyard on the next slide so you guys understand exactly what to do. So first we have the bed, the house having a backyard on the left and the house having, not having a backyard on the right. So our standard deviation, we already calculated this for you guys. It's $50,000 and the other side it's 54, roughly $54,000. And as you can see, we have two data points that have this condition. So our weight would be two over five and we multiply that by our standard deviation. Then we add this by our weight for the other side. So we have three data points out of our five. So that's our weight times our standard deviation of the house price of a house that doesn't have a backyard. And then this is our total standard deviation of this feature. Okay, does anyone have any questions? Okay, so now it's your turn to calculate the standard deviation of the bedrooms greater than three feature and the individual standard deviations have been provided to you. So just the last part, you guys don't have to calculate the standard deviation first of the bedrooms being greater than three and the bedrooms being less than three. So just put your value in the chat. I'll give you guys a few minutes to do this. Okay, I've been getting a lot of direct messages and nearly every one of you are right. Congratulations, guys. So glad. So our answer is $16,000 or roughly $17,000. And how we did this, we multiplied our weight. So we have two data points over five. So that's our weight multiplied by our standard deviation. Then we add this with our other weight. So 3 over 5 times our other standard deviation. And that's how we get our standard deviation of the whole feature. Okay, so why is this important? This is important because we will know what to split on. So our first feature being the bedrooms having less than three in our house. That's our first attribute we want to split on because it reduces the standard deviation the most. So if we go back, remember these two smaller subsets of data. So the standard deviation of these two values are a lot smaller than it was for the backyard. So that just means that if we split right here, these values are closer to each other separately. So that means it'd just be better to split on at the start. And we can repeat this process until we are done splitting and setting the averages as leaps. So as you can see, the backyard, if we split on that feature, it does not reduce standard deviation that much. So we wouldn't want to do that yet. Okay, and I'm going to ask Calvin to start to talk about generative art. Hello everyone, my name is Calvin, and so let's talk about some generative art. So generative art is a type of art where the artist uses algorithms, computer programs, or other automated systems to create art. So as you can see down below, and that is an example of generative art. And then where is generative art used? So generative art is used in art, product design, digital media, music videos, fashion design, advertisements, video games, and like the list goes on. And this is like a very broad scope, and it's something that's like, it's definitely going to be used a lot more in the future. So this is a potential like career like spot. So how does generative art work? So generative art works by taking in an algorithm, and possibly an initial output to produce an output. Okay, so today we're going to focus on supervised and unsupervised. So supervised learning algorithm, like just a quick reminder, you have data and labels, and then unsupervised learning algorithm is just data, and then reinforcement learning algorithm is like the state's action rewards. Okay, so now let's talk about GANs or generative adversarial network architecture. And these are the one type of machine learning that is used to create a generative art. And one example of this is DCGAN. So the GAN, or generative adversarial network architectures, I'm just going to refer to that whole name is just GAN from now on, because it's kind of long, but GANs consist of two models, a generator and a discriminator. And then the generator creates and the discriminator judges the creation and gives feedback. So the generator, so going a little bit more into the generators, the generator generates images from random noise using deconvolutional layers. And the discriminators inputted real and generated images and determine if the image is real or fake using a CNN. And then the generator and the discriminator are both improved during the training architecture. Um, so then once the training is complete, the generator is used to create images, like for example, generative art. And so this is how generative art is created. So once the GAN is fully trained, and both the generator and the discriminator is trained, then the generator can be used to generate images in the frame, this example generative art. So I remember we talked about loss functions, maybe one to two lessons ago, but GANs have two loss functions, one that is associated with the discriminator, and one that is associated with the generator. And then so the discriminator loss function penalizes the discriminator for incorrect slash real or fake classifications. So for example, if the discriminator says that the image is real, when it is actually fake, it gets punished, and vice versa. If it says the image is fake, but it's actually real, then it gets punished as well. Okay, and then the generator loss penalizes the generator when it is unable to fool the discriminator. So for example, if the discriminator gets it correct, it says it's a real image, when it is a real image, the generator is penalized. And when it's a fake image, it's a fake image, the generator gets penalized as well. Okay, and then backward propagation takes the loss functions into account, and then updates the weights of the model. And I think we learned this with the neural networks, and potentially this convolutional neural networks as well. And then so the loss function, so in with the GANs, and GANs are a combination of unsupervised training and supervised loss, because you have unsupervised training with the supervised loss function. Okay, so now we kind of want to do a quick chat point. I know those slides were a little bit fast, but we want to identify for both A and B if it represents code for the generator or the discriminator based on what we learned. So just a quick recap, the generator generates the images, and the discriminator tries to figure out whether the images that are inputted to the discriminator are real or fake. So I'll give you guys maybe two minutes to figure that out. Okay, um, I think a lot of you guys got that so there was some confusion But I think we can move on to the answer So Yeah, let's yeah Okay So this one is um A is the discriminator and B is the generator like many of you guys got Some of you guys had it flipped and sorry if this wasn't clear that we had Yeah So one A is the discriminator and then B is the generator and so the reason for this is the discriminator is a convolutional neural network with one input which is real or fake and so when we see that is the number of filters and the filter size and and then you see the input shape is 2828 by 1 and so this is why this A is a discriminator and D is a generator an additional an additional piece of information these codes are a little bit Oversimplified like we forgot to add the stride and padding and actually purposely deleted them as to not cause like some confusion But these codes wouldn't like likely work as is so you probably might need to know with the stride and padding as well So, yeah, so now let's talk about some of the examples of GANs So the examples of GANs is that the name of the painting is called Edmond de Bellamy Bellamy in French, which means good friend It is a twist on the last name of the creator of GANs Ian Goodfellow and this GAN created painting was sold for almost half a million And so you can see the loss function is printed in the lower right hand Corner and it signifies the loss function for GANs So now let's go on to this image so this image we see Vladimir Putin riding a horse I want to turn turn the horse into a zebra However, this is example where the GAN went wrong For example, only the horse is supposed to be converted to the zebra But instead Vladimir Putin is converted to a zebra as well, which obviously doesn't make sense because he is not a zebra We don't want to become a zebra Okay, and then another thing is where we can do a deepfake so you can see like very similar news articles and videos so we can see that There's not a video so we can see that original and deepfake So they're near similar and there's a lot of ethical implications on like generating images slash videos because anything on the internet could soon become deepfake and you wouldn't be able to distinguish between real or fake Okay, so now let's talk about DALI 3 so DALI is a generative So it's a non GAN machine learning model developed by OpenAI for generating images from text descriptions So it's an extension of ChatGPT, which I think a lot of you guys may have seen or used before So it's an extension of the ChatGPT architecture and it is supposed to be even better than GANs So The DALI 3 has images and caption vectors that are mapped to one another So they are trained using another model and so computers only work with numbers So images and captions have to be translated to the vectors So DALI 3 can take in a caption vector and produce an image so we can say that two teddy bears So for example, if I inputted a phrase saying I want Generate an image for me of two teddy bears playing with electronics under the sea an image like we can see like here Two teddy bears playing with electronics under the sea is generated. And so this is how DALI 3 works So, yeah, I think that's the lecture for today and maybe Naomi can show the homework If you I think this topic is relatively easy because it's like less math But it's still definitely an intriguing subject, but I can let Naomi take it over Yeah, and once Naomi reviews the homework I can go over some of the code and stuff since we have some extra time in the GAN section and as well as Since there's only like two more Sessions before the test and we won't have much review time We can also review anything that we have covered in the past several sessions anything that you still have doubts on But I'll let Naomi first review the homework and then we can talk more Yeah, sure, let me share my screen Okay, so here is your next homework so First question. This will be more of a decision tree question so We're given the mean and standard deviation and then we're just asked a bunch of questions on You can refer to the slides if you have any questions or the recording and Which feature should we split on? Remember that was taking the standard deviation of both classes and then figuring out which one Would reduce the standard deviation the most Then our second question just looking through a paper and it uses GANs and just Trying to understand as much as possible Trying to understand as much as possible I really recommend Reading the abstract and then you can also Look at the images and captions of the research paper. You don't have to actually read the paper but you know the more you do the more you learn papers are pretty difficult reads but Just just looking at the figures and captions that that makes a great deal in understanding And then we are going to be using this edge to cats You can just use this link to generate an image of a cat and you can make uh some funny shapes too And then you can also share your uh image on google classroom So that is your homework Awesome Awesome thanks Naomi. Naomi, do you have a link to the slides? Can you just send that to me? Maybe I can share screen and if not, then you can share screen So I'm just pulling up the code again. I'll talk you through it and know that since I am not talking through it, it could show up in our competition. Can you guys see the screen? Okay. So yeah, just to talk through this code a little bit more. So this is TensorFlow Keras. I believe that you use something similar already for the neural network code. And this is literally just the code for the convolution neural network, kind of like how Coven mentioned. We have simplified this code. There are other things like stride and padding, which we are not talking about in this class, totally outside the scope. So in theory, this code just as is wouldn't work, but it's very similar to what it would look like. And so you can see, if you remember from last time when we talked about convolution neural networks, this is just kind of our starting code. So don't worry too much about this first line, but this is the Con2D layer. And so this has a number of filters, right? Remember, if you can see my hands, the filters, they go through the image like this, right? Just like if you are taking pictures of yourself on your camera, it's the same sort of idea where the filter is passed through the entire image. And then the filter size, remember, we talked that those filters are usually like odd size, like three by three by five by five, seven by seven, you know, something odd sized and it's like square filters. And so this is just the filter size that it's a five by five square. And then the input shape is the original shape of the image, right? It's a 28 by 28 image with one channel. One channel means what? Is it an RGB image or is it grayscale? If you can write to me in the chat. I'm trying to see your faces. Yeah, I got one answer from Kitty, which is correct. Any other? Yeah, okay, lots of, there, yeah, and everyone. So this is grayscale, right? If it's just one channel, how many channels would there have to be if I told you this was an RGB image, meaning a colored image? What would that number be? Yeah, three, good memory. Three, exactly, right? RGB, red, green, blue. Then after that is ReLU. What is ReLU? And guys, this is great review, right? We've done this kind of stuff. What is ReLU? You can maybe give me the formula or you can tell me just what it is. In general, yeah, rectified linear unit. It's an activation function, great. Anyone remember the formula? Yeah, maximum of zero and X, perfect. Then you have another convolution layer, right? We saw that in our diagram from last week that you have many convolution, ReLU convolution, ReLU layers. And then finally, you just flatten it and then there's that one node at the end, which Kevin talked about, right? What does that one node identify in this case, right? We're not identifying cats or dogs, but in this case, when we're talking about GANs, what is our one node doing here at the end? What is it identifying? Whether it's exactly real or fake, real or fake, whether the image is real or fake. So this is always gonna be one at the end for a GAN. So GAN is interesting because it's literally comprised of these two models. So that is our discriminator. And then our generator is this. Now, the only difference, really major difference, I would say, is this CON2D transpose. That's essentially the deconvolution layers. So does that make sense? Does anyone have questions on this? Sorry, I have these random pop-ups coming up on my brother's laptop, so I'm trying to get them out. But yeah, if anyone has questions, put them in the chat. I did have a request from someone to review the neural network slides. Is there anything else we wanna review in GANs or did it make sense? Does the loss functions make sense and all that stuff? The generator code, quickly? Yeah, yeah, I can repeat the generator code as well. So generator code, I wouldn't worry too much about that. The only reason I'm worried more about the discriminator code is because the discriminator is nothing but a CNN, which we have already discussed. The generator, the only thing that I want you to really know is that it's sort of the opposite of that. And so you can compare these line by lines, right? You have your CON2D transpose and your ReLUs, right? Here you had, the discriminator had CON2D. You didn't have CON2D transpose. So that's the difference here. So the CON2D transpose means the deconvolution layer. It's sort of the inverse of that. Here you start off with your dense layer, whereas here you're ending with it. The numbers here might not be exactly correct and all, but I just want you to, this is just a general idea for what this should look like. But yeah, I wouldn't worry too much about the generator. The whole idea is that I want you to see that that CON2D transpose is coming. So this is not a regular convolution neural network. It has deconvolution layers. Your activation functions, we haven't looked at this one, but the activation functions might be different a little bit but I wouldn't worry too much about this. This is a little bit more. Thank you. Any other questions about any of the other topics we covered in GAN? Out of curiosity, if you had this kind of money, would you purchase this painting for half a million dollars? I got some quick responses. Kitty's like, nope. Oh, Dia was a yes. No, Rohan. Have you guys seen deepfakes before? No way, Jose, wow. You guys have seen deepfakes? No, okay, we can, let's look at one. You have, yeah. Do you remember whose deepfake? Was it like a celebrity? Yeah, I think it was Alan Becker's. Okay, interesting. He also used a deepfake for Luke Skywalker in the Mandalorian. Interesting, yeah. Let's see if we can see this Anderson Cooper one that I had. Here's the, I think. Second youngest son. He's suggesting it came out of nowhere. We subsequently learned is it may have come from the former president or his legal team acting in bad faith. This is deepfake example of what is possible with powerful computer and editing. It took around 72 hours to create this example from scratch using extremely powerful GPU. It could be improved with more computing time, but 90% of people cannot tell the difference. So hopefully you guys understood. Let me just close this. I've said many times on this show that you should trust nothing and bear. Okay, so that first one you saw, that was the real Anderson Cooper, a newscaster, right? He's telling whatever he's saying. But the second one was completely generated by a computer. He has never said those words or anything in that order. It's a deepfake. Would you have been able to distinguish if I hadn't told you that that's a deepfake or did it look pretty real? The wording, like the AI doesn't like take pauses. So you can kind of just like tell that way. Interesting. Yeah, maybe that's one way to, the AI doesn't take pauses. I think they're also too blue. His eyes were too blue when the AI did it. Yeah, yeah, his eyes looked kind of scary. Yeah, they were very blue. Darius wrote, his lips didn't match up with the words. There's a little differences in the face, Kitty said. The AI looks smoother and not very textured. Very interesting. Arha said that they saw one for Joe Biden for the election. Yeah, so that's the sort of issue for deepfake that whenever it's election season, especially, and there's a lot at stake, you will see a lot of political figures, right? Deepfakes being made of them. Yeah. So I saw an Alan Becker one. Alan Becker is a famous animator. And they took one of his most famous animations and put it in this ad and generated an AI of him. There's actually, it's actually on YouTube. You can search up ads that use Alan Becker. Yeah, if you wanna put it in the Google Classroom later. And this goes for anyone who has seen any deepfakes that they've thought of. Parents blocked YouTube, so I'm not able to. Oh no, okay. I'll see if I can find it or if anyone else finds it, feel free to kind of drop it in either the chat or Google Classroom. But yeah, it is very interesting. I think people have done deepfakes with Taylor Swift and stuff. And so there's a lot of issues like that. But yeah, any other last questions about any of the GAN topics? Let me go open up Naomi's link again. Anything else? Otherwise we can review neural networks. Okay, yeah, no more questions for that. Neural networks, people wanted me to review that. Is there anything in particular you want me to review? Okay, in the homework, okay. We have a question for this week's homework. We are supposed to take the standard deviation of a whole data set. How do we do that? That's a great question. So just to save time in class, a lot of the examples we did today, the standard deviation was provided to you. However, let me open this up again. Let me just share screen. The sort of one of the first examples we did was calculating the standard deviation, right? So over here, this formula, we calculated the mean last week, right? If you remember at the end of last week's lecture, that it was four leaves, you just add them up, divide by five, you get four. But then you had to use this equation for standard deviation. If you haven't heard of standard deviation before, the way to think about it is like, if your teacher says they grade on a curve, which you may or may not have experienced yet, but basically anyone who's average, right? If we look at this bell curve, anyone who's average, they get a C, anyone who's a little bit above average, it gets a B or a little bit below, sorry, little bit above average gets a B, little bit below average gets a D. And then anyone who's really extraordinary gets an A or really, really not that great gets an F, right? And so that's what standard deviation is. It's sort of the spread. It's this variation of the values as a variable around mean. And so what you're going to do is you use this formula to calculate the standard deviation. So you take each of these values, right? Four minus the mean bracket square plus five minus the mean bracket square plus two, so on and so forth, divided by just the total number of rows we have. So that's divided by five and you get 1.414. So in our case, you know, for example, we have each of these prices, we're going to calculate the mean and then we're going to calculate the standard deviation similarly, just like this formula. And then same thing for here. So that was provided to you here, but you would have had to calculate that like for the homework or something. And then similarly for here, for this, the standard deviation, you use these two values and you calculate the standard deviation here. You take these three values and you just plug and chug into the equation. Whenever you have two values, it's very easy to calculate standard deviation. You don't need to go through the entire formula. All you need to do is subtract the two and divide by two because your average is going to fall right in between these two. So 160 minus 60 is 100, right? So you 100 divided by two is standard deviation is 50. I won't explain right now exactly how I'm getting to it, but think about, take any two numbers and try to do the standard deviation. There's also standard deviation calculators online if you want to double check your work. I like doing that, but obviously those calculators won't be available to you online, like the specific standard deviation calculators during the time of the competition. Okay, so, but in the data set, there are three different sections. So which one do you take? Naomi, can you please give me the link to the homework? Sorry, I don't have the classroom open and I'm on my brother's laptop. Yeah, sure. Let me try to also log in here to Google Classroom. Oh, you are logged in, okay. Sorry, my brother's right here, so. Okay, so. Yeah, so Kitty, your question is about this homework, right? Calculate the standard deviation, mean standard deviation. Yeah, so you'll have to kind of go over the class example we did again. For this one, when we say for part A, you take all these values, right? But then when I was talking about part B, then you take number of classes taken is less than six. So you need to break this table into two different tables. One table will just be the rows where the number of classes taken is less than six. The other table is just when number of classes taken is greater than or equal to six. Again, for C, you repeat that, but you need to split this table into two different tables. One where the number of extracurriculars is less than five and one which is greater than equal to five. And then whatever the remaining values are in each table, you're gonna take the mean and standard deviation of that. Yeah, yeah, exactly. Okay, so your question is, what do you take the standard deviation of? Yeah, so for the grade in classes. So that's what you're trying to predict it, exactly. So I think, yeah, we're on the same page, cool. Cool, so if there's no other questions on this, I can go back and review neural networks. Dia, was that a question on the homework or is that question on the lecture material? And if anyone else also has questions, drop them in the chat, in the lecture material. Okay, Naomi, can I please ask you to share the lecture for neural network? Yeah, sure, let me get that open really quick. Do you want the link or do you want me to share my screen? Yeah, can you share the link? I'll just see if I can quickly share it. And yeah, if anyone else has any questions, please drop it in the chat. We don't go too much in depth in GANs because then the math gets super complicated, but this is a good time for reviewing because next week and the week after, there's still gonna be a lot of other hard mathematical concepts, especially in two weeks from now. So we won't have much time to review like in class for the competition. Okay, Dia, if you can tell me which kind of slide you are thinking about. That you would want me to review and go down. You said you wanted solving the model, right? So it's like kind of this section. Right, is that kind of what part you want? Okay, cool, cool, cool. Okay, yeah, we can review this. So yeah, this definitely isn't easy. And in fact, sorry, Naomi, can you give me the... I posted on Classroom the additional material for neural networks because that one has them more written out. So guys, if you haven't seen that, I posted it later where I have written it out more. It's kind of hard to see in the slide. It's always better to see something written out. Do you mean the homework key? The one where you have it written out? Yeah, I think, yeah, I have the classwork key, I think in that. I forgot what it's called. I posted on Google Classroom, like additional material for neural networks or extra info or something. Maybe I can try to open Classroom also. Okay, I think I sent you the link to the folders and in there you can... Oh, perfect, thank you. Yeah. Is this the? OK, let me try looking at. So, yeah, this is more written out, so if you guys haven't seen it and you're still confused about when your networks work, just check this out. You can see that, let me see if I can, I don't know how to do a good split screen, but basically you have these numbers and you need to match these numbers to the arrows, right? This is like 1, 1 is like, the first one is representing this thing, this circle. When it's like 2, 1 or something, then this 2 is representing this, because this is the second circle. So this is the first circle, this is the second circle. And then in the second row, it's first, second, third, right? And so 1, 1 represents this line. This one, since it goes from the second circle in the first layer to the second, sorry, to the third circle in the second layer, okay, let me do that again. It goes to the second circle in the first layer, to the third circle in the second layer, that's 2, 3. So that's how we get this, and you just had to map each of these numbers to the arrows. So far with me, and I'm not just asking Di, I'm asking everyone, because this is important stuff. Because this forms the basis of all neural networks, right, even CNNs, GANs, everything. And yeah, if Di, you can let me know if that makes sense so far, or if someone's confused and wants me to repeat it. Are we getting how these numbers are mapped to these arrows? Okay, perfect. And then the biases are just these extra things, where it's on the node, like you have bias 1 here, bias 2, bias 3, right? So the way the neural network works, right, is the first thing you need to do is you need to label each of these arrows with the numbers. It's just a little bit easier. And then you have to start, like, multiplying. So think about, like, this node, right? Think about what's going into this node. So this 3 is going into this node, this 1, so these two lines are going into this node. So what you need to do is you take the first layer, and you multiply it, like, this is the weight, right? You multiply it with the weight, and then you add it with this again. You multiply this layer times the weight, and then you add the bias. The formula of this is the weight times the input plus the, another weight times the input plus the bias, right, w1 x1 plus w2 x2 plus b, right? And so you get negative 1 times 3 plus 1 times 1 plus 0 is negative 2. Now in the second node, what are the things that are coming into it? Well, it's this and this weight, the 2 and the negative 1. So again, you had a 2 times negative 1, weight times the input plus weight times the input. So negative 1 times 1 plus the bias is 4, and that gets you 1. Now again, for the third node, you think about what is going into that. Then it's this negative 2 that's going into it, and it's this 0. So you had to multiply negative 2 times the input, so negative 1, plus 0 times 1, plus our bias, 1, and then that gives you 3. So far with me, everyone, does that make sense? Or if anyone has questions, stop me. We have like six more minutes, so I'm going to try to wrap it up. Okay, cool. So that's what you're getting essentially for each of these. That's your output, negative 2, 1, 3. Now, this was the simplified version. This is where I didn't define any activation function, so you assume that it's a linear activation function. But now in regular neural networks, you are going to have some activation function. So whatever this output you get, you still need to do further processing, right? So right now, these are all linear, so you can say this is the output. But what if I told you all of these are ReLU? Can we tell me the formula of ReLU in the chat? And I know we did this earlier today, so hopefully I get more answers now. So what's the formula for ReLU? Yeah, maximum of 0 and x, right? Maximum of 0 and that value. So now you can't just say the output is negative 2, 1, 3. You take each of these negative 2, 1, 3, and you take the maximum of 0 and that thing. So what's the maximum of 0 and negative 2? 0, right? What's the maximum of 0 and 1? 1. What's the maximum of 0 and 3? 3, right? So now your new output is 0, 1, 3, if ReLU was all of our activation functions, right? And so I think, yeah, we have an example like this over here, right? How would you calculate this? Well, there's two weights going into here, 3 times 2 plus 1 times negative 2 plus your bias. So that's 4. But then there is a ReLU activation function defined here. So you have to do maximum of 0, 4, and you get 4. And so that's your output. And then hinge loss or any loss, you just look at the formula and you just plug and chug. And your t is your target value. That means that's what it should have come out as. And your y is your prediction. That means that's what the neural network is giving you. And so the problem will tell you what you expect to get, right? So the problem might say, oh, we expect to get 3. But instead, our neural network is giving us a 4. And so we look at the hinge loss and we see if that's a problem or not, right? And so if you do that, it's maximum of 0, 1 minus target 3, 4. So that's 1 minus 12. That's negative 11. The maximum of 0 and negative 11 is 0. So that means there is no loss. There is no error. In certain models, you know, as long as the sign is same, meaning expected output and our predicted output, the sign is same, they're both positive. As long as the sign is same, then there's no issue, right? There's no error as per the model. That's actually what, and this is extra information, but hinge loss. Hinge loss is always 0 when the sign of predicted and actual is same because you just want the signs to be the same. You don't want it to necessarily be the exact, exact same number. Does that make more sense of how neural networks are calculated? Or are there any other questions from anyone? Loss is just the error. That's the way to think about it. What is your neural network trying to give you? What's the predicted value? And what is the actual thing that you are expecting? Any other questions or did this all make sense now? Diya, did this make sense or did you have questions for Nirla or anyone else? Awesome. Any other, we have two more minutes, any other last questions for decision three regression or classification, intro to AI, intro to ML? We have done lots and lots of math, right? This is going to be a mathematically intensive competition, so can quickly review your last few assignments, see if you have anything. Will we be expected to, yeah, in the competition, will we be expected to know the formulas such as hinge loss and standard deviation or will they be given? That's a good question. I haven't decided yet. Especially since it's open book, I'm of the opinion that I don't really feel the need to give it to you because it's going to be open book in the sense that you can have a cheat sheet. You can have your notes. You can't use like the lecture slides or you can't use the internet or anything like that. But since you can use your notes, what I would say is write down the formulas yourself. I don't, if I need to give you a formula, absolutely, then I will, otherwise I won't. For something for standard deviation, especially, I'm not going to give you that. I don't think so. Yeah. So just assume I'm not going to. If I feel very charitable for some reason, then I might, but let's assume that for at least the common things, I won't give you formulas. Anything that's taught in class, I don't expect you to memorize it. I don't have them memorized either, but I expect them for you to have it in your notes or even better yet, just have a one page cheat sheet, right? Having 15 million page notebook, like that doesn't really help. Cheat sheets is what we did at MIT and I thought that that was helpful because that gives you extra time to like prepare and memorize the content and stuff. But that's a good question. Yeah. In fact, you know what? I'll just say for now, I'm not going to give you, I'm not going to give you formulas for anything we covered in class for sure. Cause I'm just thinking about how I would write that on a Google form and it's just going to take too long. So since we've already covered in class, just write it in your notes. If there is a formula of something like we haven't covered in class, for example, like a type of loss function that I haven't covered, like you obviously are not expected to know from the top of your head and I would give you that formula and you would just use it to plug and chug, right? The competition is not going to cover any material that you don't already know or that you can easily calculate it based on stuff we have covered. One other pro tip I'm going to say about the competition is that it's going to start combining different concepts from different classes, right? Which the way we teach the classes, each is a singular unit, but these are all interconnected topics. So I would kind of keep that in mind. Yeah. You are definitely allowed to use calculators, but your own, like not an online calculator because the internet won't be allowed. Any other last questions? We are now out of time. So if you don't, enjoy the rest of your weekend and we'll see you next week.

machine learning

neural networks

average pooling

dot product

Google Drive

training process

image classification

generative art

decision trees