Error converting content: marked is not a function
- TODO Lesson 0 - How to fastai collapsed:: true - {{video https://www.youtube.com/watch?v=gGxe2mN3kAg}} - [Lesson 1 - Getting started](https://course.fast.ai/Lessons/lesson1.html) collapsed:: true - {{video https://www.youtube.com/watch?v=8SF_h3xF3cE}} - - YT Chapters collapsed:: true - [00:00](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=0s) - Introduction [00:25](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=25s) - What has changed since 2015 [01:20](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=80s) - Is it a bird [02:09](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=129s) - Images are made of numbers [03:29](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=209s) - Downloading images [04:25](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=265s) - Creating a DataBlock and Learner [05:18](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=318s) - Training the model and making a prediction [07:20](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=440s) - What can deep learning do now [10:33](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=633s) - Pathways Language Model (PaLM) [15:40](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=940s) - How the course will be taught. Top down learning [19:25](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=1165s) - Jeremy Howard’s qualifications [22:38](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=1358s) - Comparison between modern deep learning and 2012 machine learning practices [24:31](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=1471s) - Visualizing layers of a trained neural network [27:40](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=1660s) - Image classification applied to audio [28:08](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=1688s) - Image classification applied to time series and fraud [30:16](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=1816s) - Pytorch vs Tensorflow [31:43](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=1903s) - Example of how Fastai builds off Pytorch (AdamW optimizer) [35:18](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2118s) - Using cloud servers to run your notebooks (Kaggle) [38:45](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2325s) - Bird or not bird? & explaining some Kaggle features [40:15](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2415s) - How to import libraries like Fastai in Python [40:42](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2442s) - Best practice - viewing your data between steps [42:00](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2520s) - Datablocks API overarching explanation [44:40](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2680s) - Datablocks API parameters explanation [48:40](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2920s) - Where to find fastai documentation [49:54](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=2994s) - Fastai’s learner (combines model & data) [50:40](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3040s) - Fastai’s available pretrained models [52:02](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3122s) - What’s a pretrained model? [53:48](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3228s) - Testing your model with predict method [55:08](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3308s) - Other applications of computer vision. Segmentation [56:48](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3408s) - Segmentation code explanation [58:32](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3512s) - Tabular analysis with fastai [59:42](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3582s) - show_batch method explanation [1:01:25](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3685s) - Collaborative filtering (recommendation system) example [1:05:08](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3908s) - How to turn your notebooks into a presentation tool (RISE) [1:05:45](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=3945s) - What else can you make with notebooks? [1:08:06](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=4086s) - What can deep learning do presently? [1:10:33](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=4233s) - The first neural network - Mark I Perceptron (1957) [1:12:38](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=4358s) - Machine learning models at a high level [1:18:27](https://www.youtube.com/watch?v=8SF_h3xF3cE&t=4707s) - Homework - Lesson Summary in Questions collapsed:: true - Welcome to Part 1 2022 course - Were computers smart enough to determine photos of birds before 2015? - How to download and display a photo of a bird from DuckDuckGo using simple codes? - What photos/images are actually made of, at least for computers? - How to create two folders named ‘bird’ and ‘forest’ respectively under a larger folder ‘dest’? How to download 200 images for each category? How to resize and save those images in respective folders? - How to find broken images and then remove or unlink them from their folders? - How to create a DataBlock which prepares all the data for building models? How to display the images in a batch? - How to build a model and train/finetune it on your local computer? - How to predict or classify a photo of bird with a model? - How to get started running and playing around the codes and models immediately and effortlessly? - Why should you read lecture questionnaires before studying the lecture? - How do you search and locate a particular moment inside a lecture video? - Can you create an original masterpiece painting by simply utterring some artistic words? - Can you believe that models today can explain your math problems not just give you a correct answer? Can you believe that models today can help you get a joke? - Jeremy and fastai community make serious effort in help beginners continuously. - Do you want to know how to make the most out of fastai? - Do you know people learn naturally (better) with context rather than by theoretical curriculum? Do you want this course to make you a competent deep learning practitioner by context and practical knowledge? If you want theory from ground up, go to part 2 fastai 2019 - Do you know that learning the same thing in different ways betters understanding? - Why you must take this course very seriously? (Personally, I think it’s truly a privilege to be taught by Jeremy and to be part of the fastai family. I didn’t appreciate it enough as I should 4 years ago.) - Why did we need so many scientists from different disciplines to collaborate for many years in order to design a successful model before deep learning? - Why can deep learning create a model to tell bird from forest photos in 2 minute which was the impossible before 2015? Would you like to see how much better/advanced/complex are the features discovered by deep learning than groups of interdisciplinary scientists? - Are all things are data, sound, time (series), movement? Are images are just one way of expressing data? Why not store or express data (of sound, time, movement) in the form of images? Can imaged based algos learn on those images no matter how weird they appear to humans? - Can I do DL with no math (I mean with high school math)? Can I train DL models with hand-made data (<50 samples)? Can I train state of art models for free (literally)? - Which should I invest my life in DL software field, Pytorch or Tensorflow? - Why should you use fastai over pure pytorch? Don’t you want to write less code, make less error, achieve better result? Don’t you want a robust and simple tool used by your future colleagues and bosses? - Why is jupyter notebook the most loved and tested coding tool for DL? Do you want Jeremy to show you how to use Jupyter notebook hand by hand? - How to make sure your notebook is connected in the cloud? How to make sure you are using the latest updated fastai? #best-practice - Doesn’t fastai feel like python with best practices too? How to import libraries to download images? How to create and display a thumbnail image? Always view your data at every step of building a model #best-practice How to download and resize images? Why do we resize images? #best-practice - Why a real world DL practitioner spend most of the valuable/productive time preparing data rather than tweaking models? Can super tiny amount of models solve super majority of practical problems in the world? Have fastai selected and prepared the best models for us already? - Does Jeremy add best practices of other programming languages into fastai? Jeremy loves functional programming - How fastai design team decide what tasks should DataBlock do? task 1: Which blocks of data do DataBlock need to prepare for training? task 2: How should DataBlock get those data, or by what function/tool? task 3: Should we always ask DataBlock to keep a section of data for validation? task 4: Which function or method should DataBlock use to get label for y? task 5: Which transformation should DataBlock apply to each data sample? task 6: Does dataloader do the above tasks efficiently by doing them in thousands of batches at the same time with the help of GPUs? - What is the most efficient way of finding out how to use e.g., DataBlock properly? How to learn DataBlock thoroughly? - What do you give to a learner, e.g., vision_learner? - Is fastai the first and only framework implement TIMM? Can you use any model from TIMM in your project? Where can you learn more of TIMM? - What is a pretrained model, Resnet18? What did this model learn from? What come out of this model’s learning? or what is Kaggle downloading exactly? - What exactly does fine tuning do to the pretrained model? What does fine-tuning want the model learn from your dataset compared with the pretrained dataset? - How to use the fine tuned model to make predictions? - Can we fine tune pretrained CV models to tell us the object each and every pixel on a photo belong to? - Why do we need specialized DataLoaders like SegmentationDataLoaders given DataBlock? - What can tabular analysis do? Can we use a bunch of columns to predict another column of a table? How do you download all kinds of dataset for training easily with fastai? untar_data What are the parameters for TabularDataLoaders? What is the best practice show_batch of fastai learned from Julia (another popular language)? Why to use fit_one_cycle instead of fine_tune for tabular dataset? - Can we use collaborative filtering to make movie recommendations for users? How does recommendation system work? Can collaborative filtering models learn from data of similar music users and recommend/predict music for new users based on how similar they are to existing users? - How to download dataset for collaborative filtering models? How to use CollabDataLoaders? How to build a collaborative filtering model with collab_learner? What is the best practice for setting y_range for collab_learner? #best-practice If in theory no reason to use pretrained collab models, and fine_tune works as good as fit or fit_one_cycle, any good explanations for it? #question How to show results of this recommendation model using show_results? - What can Deep Learning do at the present? What are the tasks that deep learning may not be good at? - Has the basic idea of deep learning changed much since 1959? - What did we write into programs/models before deep learning? How to draw chart in jupyter notebook? - What is a model? What are weights? How do data, weights and model work together to produce result? Why are the initial results are no good at all? Can we design a function to tell the model how good it is doing? loss function Then can we find a way to update/improve weights by knowing how bad/good the model is learning each time from the data? If we can iterate the cycle multiple times, can we build a powerful model? - Homework: Run notebooks, especially the bird notebook. Create something interesting to you based on the bird notebook. Read the first chapter of the book. Be inspired by all the amazing student projects. - [](https://course.fast.ai/Lessons/lesson8a.html) - Notes collapsed:: true - Wow - feature engineering as ML back int he day - {:height 242, :width 557} - Resources - [timm pytorch](https://timm.fast.ai) - [Lesson 2 - Deployment](https://course.fast.ai/Lessons/lesson2.html) collapsed:: true - {{video https://www.youtube.com/watch?v=F4tvM4Vb3A0}} - YT Chapters collapsed:: true - [00:00](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=0s) - Introduction [00:55](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=55s) - Reminder to use the fastai book as a companion to the course [02:06](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=126s) - aiquizzes.com for quizzes on the book [02:36](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=156s) - Reminder to use fastai forums for links, notebooks, questions, etc. [03:42](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=222s) - How to efficiently read the forum with summarizations [04:13](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=253s) - Showing what students have made since last week [06:45](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=405s) - Putting models into production [08:10](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=490s) - Jupyter Notebook extensions [09:49](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=589s) - Gathering images with the Bing/DuckDuckGo [11:10](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=670s) - How to find information & source code on Python/fastai functions [12:45](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=765s) - Cleaning the data that we gathered by training a model [13:37](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=817s) - Explaining various resizing methods [14:50](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=890s) - RandomResizedCrop explanation [15:50](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=950s) - Data augmentation [16:57](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1017s) - Question: Does fastai's data augmentation copy the image multiple times? [18:30](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1110s) - Training a model so you can clean your data [19:00](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1140s) - Confusion matrix explanation [20:33](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1233s) - plot_top_losses explanation [22:10](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1330s) - ImageClassifierCleaner demonstration [25:28](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1528s) - CPU RAM vs GPU RAM (VRAM) [27:18](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1638s) - Putting your model into production [30:20](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1820s) - Git & Github desktop [31:30](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=1890s) - For Windows users [37:00](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2220s) - Deploying your deep learning model [37:38](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2258s) - Dog/cat classifier on Kaggle [38:55](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2335s) - Exporting your model with learn.export [39:40](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2380s) - Downloading your model on Kaggle [41:30](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2490s) - How to take a model you trained to make predictions [43:30](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2610s) - learn.predict and timing [44:22](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2662s) - Shaping the data to deploy to Gradio [45:47](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2747s) - Creating a Gradio interface [48:25](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=2905s) - Creating a Python script from your notebook with #|export [50:47](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=3047s) - Hugging Face deployed model [52:12](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=3132s) - How many epochs do you train for? [53:16](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=3196s) - How to export and download your model in Google Colab [54:25](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=3265s) - Getting Python, Jupyter notebooks, and fastai running on your local machine [1:00:50](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=3650s) - Comparing deployment platforms: Hugging Face, Gradio, Streamlit [1:02:13](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=3733s) - Hugging Face API [1:05:00](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=3900s) - Jeremy's deployed website example - tinypets [1:08:23](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=4103s) - Get to know your pet example by aabdalla [1:09:44](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=4184s) - Source code explanation [1:11:08](https://www.youtube.com/watch?v=F4tvM4Vb3A0&t=4268s) - Github Pages - Summary collapsed:: true - New exciting content to come - Can there be substantial new content given we have already 4 versions and a book? - Ways of reading the book - How many channels available for us to read the book? (physical, github, colab and others) - Extra sweets from the book - Are there interesting materials/stories covered by the book not the lecture? - Where can you find questionnaires and quizzes of the lectures? - aiquizzes.com - Where can you get more quizzes of fastai and memorize them forever? - Introducing the forum - How to make the most out of fastai forum? - Students’ works after week 1 - A Wow moment - Will we learn to put model in production today? - Find a problem and some data - What is the first step before building a model? - Access to the magics of Jupyter notebook - Do you want to navigate the notebook with a TOC? - How about collapsable sections? - How about moving between start and end of sections fast? - How to install jupyter extensions - Download and clean your data - Why use ggd rather than bing for searching and downloading images? - How to clean/remove broken images? - Get to docs quickly - How to get basic info, source code, full docs on fastai codes quickly? - Resize your data before training - How can you specify the resize options to your data? - Why should we always use RandomResizedCrop and `aug_transforms` together? - How RandomResizedCrop and `aug_transforms` differ? - Data images instantly transformed not copied - When resized, are we making many copies of the image? - More epochs for fancy resize - How many epochs do we usually go when using RandomResizedCrop and `aug_transforms`? - Confusion matrix: where do models get wrong the most? - How to create confusion matrix on your model performance? - When to use confusion matrix? (category)-practice - How to interpret confusion matrix? - What is the most obvious thing does it tell us? - How hard is it to tell grizzly and black bears apart? - Check out images with worse predictions - Do `plot_top_losses` give us the images with highest losses? - Are those images merely ones the model made confidently wrong prediction?-practice - Do those images include ones that the model made right prediction unconfidently? - What does looking at those high loss images help? (get expert examination or simple data cleaning) - What if you want to clean the data a little - How to display and make cleaning choices on each of those top loss images in each data folder?-practice - Without expert knowledge on telling apart grizzly and black bears, at least we can clean images which mess up teddy bears. - **Myth breaker: train model and then clean data** - How can training the model help us see the problem of dataset?-practice - Won’t we have more ideas to improve the dataset once we spot the problems of the dataset? - Turn off GPU when not using - How to use GPU RAM locally without much trouble? - Watch first, then watch and code along - What is the preferred way of lecture watching and coding by majority of students? - A Gradio + hugging face tutorial - Git and Github desk - Is Github desk a less cool but easier and more robust way to version control than git? - Terminal for windows - How to set up terminal for windows? - Why Jeremy prefer windows than mac? - Get started with Hugging Face Spaces - go to huggingface.co/spaces and create a new space - Get the default App up and running - How to use git to download your space folder? - How to open vscode to add app.py file? - How to use vscode to push your space folder up to hugging face spaces online? - then go back to your space on Hugging Face to see the app running - **Train and download your model** - Where is the model we are going to train and download from Kaggle notebook? - How to export your model after trained it on Kaggle? - Where do you download the model? - How to open a folder in terminal? `open .` - Make sure the model is downloaded into its own Hugging Face Space folder - Predict with loaded model - How to load downloaded model to make prediction? - How to make prediction with the loaded model? - How to export selected cells of a jupyter notebook into a python file? - How to see how long a code runs in a jupyter cell? - Turn your model into Gradio App locally - How to prepare your prediction result into a form gradio prefers? #code - How to build a gradio interface for your model? - How to launch your app with the model locally? - Not in video: run the code on Kaggle in cloud - Push this app onto Hugging Face Spaces - Make sure to create a new space first, e.g., testing - How to turn the notebook into a python script? - How to push the folder up to github and run app in cloud? - Not in Video: if stuck, check out Tanishq tutorial-shooting - How many epochs are ideal for fine tuning? - How to save model from colab? - How to install fastai properly - How to download github/fastai/fastsetup using git? `git clone https://github.com/fastai/fastsetup.git` - How to download and install mamba? `./setup_conda.sh` - Not in Video: problem of running `./setup_conda.sh` - How to download and install fastai? `mamba install -c fastchan fastai` - How to install nbdev? `mamba install -c fastchan nbdev` - How to start to use jupyter notebook? `jupyter notebook --no-browser` - Not in Video: other problem related to xcode - The workflow summary - HuggingFace API + gradio + Javascript = real APP - How easy does HuggingFace API work - How easy to to get started with JS + HF API + gradio - App example of having multiple inputs and outputs - App example of combining two models - How to turn your model into your own web App with fastpages - How to fork a public fastpages for your own use - My Timestamp Notes - {{youtube-timestamp 648}} jupyter stuff and overview - {{youtube-timestamp 778}} before you clean the data, you train the model - {{youtube-timestamp 1432}} nice notebook ui to see images and clean data - {{youtube-timestamp 2409}} ok trained a model on collab and/or keggle, then exported it; then uploaded it to huggingface to use via gradio ui; good product readyness application end to end experience - {{youtube-timestamp 2488}} ok, now prediction on saved model - {{youtube-timestamp 2943}} `#| export` - to export code into script from notebook - app deployment on huggingface - {{youtube-timestamp 3291}} running on local machine - {{youtube-timestamp 3439}} break - moving to conda instead of base python for AI - {{youtube-timestamp 4596}} last part of fun quick web site deployment using fastwebsite and github pages - Lesson Resources and Links - Important: [Gradio + Huggingface tutorial](https://tmabraham.github.io/blog/gradio_hf_spaces_tutorial) from @ilovescience - @Tanishq Abrahams fast.ai member child prodigy collapsed:: true - {{video https://www.youtube.com/watch?v=tMEcVQ23X6c}} - [Forum](https://forums.fast.ai/t/lesson-2-official-topic/96033) - Notebook—saving a basic fastai model: collapsed:: true - [Kaggle](https://www.kaggle.com/jhoward/saving-a-basic-fastai-model) - [Colab](https://colab.research.google.com/drive/1M-mzhZdFQ2XWBSbLCuKzrmLsm0aLEYxQ?usp=sharing) - Chapter 2 [notebook](https://github.com/fastai/fastbook/blob/master/02_production.ipynb) - [Solutions](https://forums.fast.ai/t/fastbook-chapter-2-questionnaire-solutions-wiki/66392) to chapter 2 questions from the book - [HF Spaces](https://huggingface.co/spaces) - Installing a python environment [fastsetup](https://github.com/fastai/fastsetup) - tinypets [github](https://github.com/fastai/tinypets) / [site](https://fastai.github.io/tinypets/) - tinypets fork [github](https://github.com/jph00/tinypets) / [site](https://jph00.github.io/tinypets/) - [Lesson 3 - Neural net foundations](https://course.fast.ai/Lessons/lesson3.html) collapsed:: true - {{video https://www.youtube.com/watch?v=hBBOjCiFcuo&embeds_referring_euri=https%3A%2F%2Fcourse.fast.ai%2F&feature=emb_title}} - YT Chapter collapsed:: true - [00:00](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=0s) Introduction and survey [01:36](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=96s) "Lesson 0" How to fast.ai [02:25](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=145s) How to do a fastai lesson [04:28](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=268s) How to not self-study [05:28](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=328s) Highest voted student work [07:56](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=476s) Pets breeds detector [08:52](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=532s) Paperspace [10:16](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=616s) JupyterLab [12:11](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=731s) Make a better pet detector [13:47](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=827s) Comparison of all (image) models [15:49](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=949s) Try out new models [19:22](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=1162s) Get the categories of a model [20:40](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=1240s) What’s in the model [21:23](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=1283s) What does model architecture look like [22:15](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=1335s) Parameters of a model [23:36](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=1416s) Create a general quadratic function [27:20](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=1640s) Fit a function by good hands and eyes [30:58](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=1858s) Loss functions [33:39](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=2019s) Automate the search of parameters for better loss [42:45](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=2565s) The mathematical functions [43:18](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=2598s) ReLu: Rectified linear function [45:17](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=2717s) Infinitely complex function [49:21](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=2961s) A chart of all image models compared [52:11](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=3131s) Do I have enough data? [54:56](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=3296s) Interpret gradients in unit? [56:23](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=3383s) Learning rate [1:00:14](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=3614s) Matrix multiplication [1:04:22](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=3862s) Build a regression model in spreadsheet [1:16:18](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=4578s) Build a neuralnet by adding two regression models [1:18:31](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=4711s) Matrix multiplication makes training faster [1:21:01](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=4861s) Watch out! it’s chapter 4 [1:22:31](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=4951s) Create dummy variables of 3 classes [1:23:34](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=5014s) Taste NLP [1:27:29](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=5249s) fastai NLP library vs Hugging Face library [1:28:54](https://www.youtube.com/watch?v=hBBOjCiFcuo&t=5334s) Homework to prepare you for the next lesson - My Timestamp collapsed:: true - First 5 mins - great tips on how to learn and use this course - {{youtube-timestamp 368}} examples from community - {{youtube-timestamp 539}} paperscape - {{youtube-timestamp 830}} improving a model by IMPROVING THE ARCHITECTURE - {{youtube-timestamp 1410}} How to NN really work? - {{youtube-timestamp 2821}} #OMG moment - a bunch Tech/ReLU together can make any #swiggly i.e the universe #boom - {{youtube-timestamp 3084}} Q/A - try fast model first, better arch is the last thing - - Lesson Resources collapsed:: true - [lesson notebook - 04- how does nn work](https://github.com/fastai/course22/blob/master/04-how-does-a-neural-net-really-work.ipynb) - [Chapter 10 fastbook](https://github.com/fastai/fastbook/blob/master/10_nlp.ipynb) - [HuggingFace Spaces Pets repository](https://huggingface.co/spaces/jph00/pets/tree/main) - [Which image models are best?](https://www.kaggle.com/code/jhoward/which-image-models-are-best/) - [How does a neural net really work?](https://www.kaggle.com/code/jhoward/how-does-a-neural-net-really-work) - Titanic spreadsheet: see the [course repository](https://github.com/fastai/course22) - Titanic data (training CSV) can be downloaded from [Kaggle](https://www.kaggle.com/competitions/titanic/) - [Solutions](https://forums.fast.ai/t/fastbook-chapter-4-questionnaire-solutions-wiki/67253) to chapter 4 questions from the book - [Lesson 4 - Natural Language (NLP)](https://course.fast.ai/Lessons/lesson4.html) - very diff from the book collapsed:: true - {{video https://www.youtube.com/watch?v=toUgBQv1BT8&embeds_referring_euri=https%3A%2F%2Fcourse.fast.ai%2F&feature=emb_title}} - YT Chapters collapsed:: true - [00:00:00](https://www.youtube.com/watch?v=toUgBQv1BT8&t=0s) - Using Huggingface [00:03:24](https://www.youtube.com/watch?v=toUgBQv1BT8&t=204s) - Finetuning pretrained model [00:05:14](https://www.youtube.com/watch?v=toUgBQv1BT8&t=314s) - ULMFit [00:09:15](https://www.youtube.com/watch?v=toUgBQv1BT8&t=555s) - Transformer [00:10:52](https://www.youtube.com/watch?v=toUgBQv1BT8&t=652s) - Zeiler & Fergus [00:14:47](https://www.youtube.com/watch?v=toUgBQv1BT8&t=887s) - US Patent Phrase to Phase Matching Kaggle competition [00:16:10](https://www.youtube.com/watch?v=toUgBQv1BT8&t=970s) - NLP Classification [00:20:56](https://www.youtube.com/watch?v=toUgBQv1BT8&t=1256s) - Kaggle configs, insert python in bash, read competition website [00:24:51](https://www.youtube.com/watch?v=toUgBQv1BT8&t=1491s) - Pandas, numpy, matplotlib, & pytorch [00:29:26](https://www.youtube.com/watch?v=toUgBQv1BT8&t=1766s) - Tokenization [00:33:20](https://www.youtube.com/watch?v=toUgBQv1BT8&t=2000s) - Huggingface model hub [00:36:40](https://www.youtube.com/watch?v=toUgBQv1BT8&t=2200s) - Examples of tokenized sentences [00:38:47](https://www.youtube.com/watch?v=toUgBQv1BT8&t=2327s) - Numericalization [00:41:13](https://www.youtube.com/watch?v=toUgBQv1BT8&t=2473s) - Question: rationale behind how input data was formatted [00:43:20](https://www.youtube.com/watch?v=toUgBQv1BT8&t=2600s) - ULMFit fits large documents easily [00:45:55](https://www.youtube.com/watch?v=toUgBQv1BT8&t=2755s) - Overfitting & underfitting [00:50:45](https://www.youtube.com/watch?v=toUgBQv1BT8&t=3045s) - Splitting the dataset [00:52:31](https://www.youtube.com/watch?v=toUgBQv1BT8&t=3151s) - Creating a good validation set [00:57:13](https://www.youtube.com/watch?v=toUgBQv1BT8&t=3433s) - Test set [00:59:00](https://www.youtube.com/watch?v=toUgBQv1BT8&t=3540s) - Metric vs loss [01:01:27](https://www.youtube.com/watch?v=toUgBQv1BT8&t=3687s) - The problem with metrics [01:04:10](https://www.youtube.com/watch?v=toUgBQv1BT8&t=3850s) - Pearson correlation [01:10:27](https://www.youtube.com/watch?v=toUgBQv1BT8&t=4227s) - Correlation is sensitive to outliers [01:14:00](https://www.youtube.com/watch?v=toUgBQv1BT8&t=4440s) - Training a model [01:19:20](https://www.youtube.com/watch?v=toUgBQv1BT8&t=4760s) - Question: when is it ok to remove outliers? [01:22:10](https://www.youtube.com/watch?v=toUgBQv1BT8&t=4930s) - Predictions [01:25:30](https://www.youtube.com/watch?v=toUgBQv1BT8&t=5130s) - Opportunities for research and startups [01:26:16](https://www.youtube.com/watch?v=toUgBQv1BT8&t=5176s) - Misusing NLP [01:33:00](https://www.youtube.com/watch?v=toUgBQv1BT8&t=5580s) - Question: isn’t the target categorical in this case? - My timestamps - {{youtube-timestamp 138}} huggingface transformers - {{youtube-timestamp 412}} ULMFit - this is where Jeremy starts in Hacker's way to LLM - {{youtube-timestamp 575}} transformers and other high level stuff - skip worthy - {{youtube-timestamp 912}} Getting started with NLP for absolute beginners - {{youtube-timestamp 1455}} how to approach comptetition, looking at data - {{youtube-timestamp 1789}} created datastet and now we tokenize - {{youtube-timestamp 1992}} tokens depends on model we use; we will use hf library to get token - {{youtube-timestamp 2114}} deberta is a good starting model generally; we learn what's a good model via competitions - {{youtube-timestamp 2543}} our document representation was arbritraty - THE MOST IMPORTANT IDEA IN Tech/ML is - Training, Validation and Test datasets. To avoid Overfitting - control the #swiggly to not be swiggly :). Also remember - Overfitting is hard to recognize. - Therefore, [creating a good validation](https://www.fast.ai/posts/2017-11-13-validation-sets.html) is Good Engineering - critical - {{youtube-timestamp 3196}} great insight and #aha into why random validation can screw ya - {{youtube-timestamp 3451}} Test set - another check - {{youtube-timestamp 3565}} METRICS - that's how we see our farming - {{youtube-timestamp 3659}} why accuracy is not the good metric to optimize - it's not smooth - {{youtube-timestamp 3712}} Problem with metrics - https://www.fast.ai/posts/2019-09-24-metrics.html - WHEN MESURE BECOMES A TARGET IT'S NOT A GOOD MEASURE #boom #metrics - {{youtube-timestamp 3875}} Pearson correlation coeffecient - https://en.wikipedia.org/wiki/Pearson_correlation_coefficient - training - {{youtube-timestamp 4830}} how to remove outlier - - - Lesson Resources - Notebook: [Getting started with NLP for absolute beginners](https://www.kaggle.com/code/jhoward/getting-started-with-nlp-for-absolute-beginners) - [Lesson 5 - From-scratch model](https://course.fast.ai/Lessons/lesson5.html) id:: 65119c1e-639f-460a-9738-84c5010f5bf1 - AWESOME - {{video https://www.youtube.com/watch?v=_rXzeWq4C6w}} - https://www.kaggle.com/code/jhoward/linear-model-and-neural-net-from-scratch - This lesson is based partly on [chapter 4](https://github.com/fastai/fastbook/blob/master/04_mnist_basics.ipynb) and [chapter 9](https://github.com/fastai/fastbook/blob/master/09_tabular.ipynb) of the [book](https://www.amazon.com/Deep-Learning-Coders-fastai-PyTorch/dp/1492045527). - gj - #revisit - https://huggingface.co/blog/fastai - Global Resource - [Fast.ai book](https://github.com/fastai/fastbook) - [Course Github](https://github.com/fastai/course22) - Fastsetup - [Fast Blogging with Quarto](https://nbdev.fast.ai/tutorials/blogging.html) - The Drivetrain Approach - [AI Quizzes](https://aiquizzes.com) - [MatrixMultiplication Visual](http://matrixmultiplication.xyz) - [How (and why) to create a good validation set](https://www.fast.ai/posts/2017-11-13-validation-sets.html) - [The problem with metrics is a big problem for AI](https://www.fast.ai/posts/2019-09-24-metrics.html)