{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Quickstart Tutorial\n", "\n", "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/masterfulai/masterful-docs/blob/main/notebooks/tutorial_quickstart.ipynb)        [![Download](images/download.png)](https://masterful-public.s3.us-west-1.amazonaws.com/933013963/latest/tutorial_quickstart.ipynb)[Download this Notebook](https://masterful-public.s3.us-west-1.amazonaws.com/933013963/latest/tutorial_quickstart.ipynb) \n", "\n", "In this short introduction to the Masterful AutoML Platform, you will train a model start to finish with Masterful. This example will walk you through creating a simple dataset and model, and using that dataset and model with the Masterful AutoML platform. This example is intended to demonstrate workflows, not train a state of the art model. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Prerequisites\n", "\n", "Please follow the Masterful installation instructions [here](../tutorials/tutorial_installation.md) in order to run this Quickstart." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Imports\n", "\n", "First, import the necessary libraries and register the Masterful package. " ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Loaded Masterful version 0.4.0. This software is distributed free of\n", "charge for personal projects and evaluation purposes.\n", "See http://www.masterfulai.com/personal-and-evaluation-agreement for details.\n", "Sign up in the next 45 days at https://www.masterfulai.com/get-it-now\n", "to continue using Masterful.\n" ] } ], "source": [ "import tensorflow as tf\n", "import tensorflow_datasets as tfds\n", "\n", "import masterful\n", "masterful = masterful.register()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup Training Data\n", "\n", "In this example, you will use the CIFAR-10 dataset from [Tensorflow Datasets](https://www.tensorflow.org/datasets). You are only going to use 5% of the labeled training data to speed up the overall training time and focus on the end-to-end scenario, rather than particular results." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "NUM_CLASSES=10\n", "TRAINING_PERCENTAGE=5\n", "(training_dataset, test_dataset) = tfds.load('cifar10', \n", " as_supervised=True, \n", " split=[f'train[:{TRAINING_PERCENTAGE}%]', 'test'], \n", " with_info=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Masterful works best with dense labels, so apply a one-hot encoding to the sparse labels provided by Tensorflow dataset." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def sparse_to_dense(image, label):\n", " label = tf.cast(label, tf.int32)\n", " one_hot_label = tf.one_hot(label, depth=NUM_CLASSES)\n", " return image, one_hot_label\n", "\n", "labeled_dataset = training_dataset.map(sparse_to_dense, num_parallel_calls=tf.data.AUTOTUNE)\n", "test_dataset = test_dataset.map(sparse_to_dense, num_parallel_calls=tf.data.AUTOTUNE)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create the Model\n", "\n", "Build a simple model to classify CIFAR-10 data. This is a toy model for demonstration purposes only, and should not be used in a production environment. It has a few convolutional layers and outputs logits directly, rather than using a softmax layer at the end. Outputting logits directly allows Tensorflow the ability to use a more numerically stable version of some loss functions." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "def get_model():\n", " model = tf.keras.models.Sequential()\n", " model.add(tf.keras.layers.experimental.preprocessing.Rescaling(1./255,\n", " input_shape=(32, 32, 3)))\n", " model.add(tf.keras.layers.Conv2D(16, (3, 3), activation='relu',))\n", " model.add(tf.keras.layers.MaxPooling2D((2, 2)))\n", " model.add(tf.keras.layers.Conv2D(32, (3, 3), activation='relu',))\n", " model.add(tf.keras.layers.MaxPooling2D((2, 2)))\n", " model.add(tf.keras.layers.Conv2D(64, (3, 3), activation='relu',))\n", " model.add(tf.keras.layers.MaxPooling2D((2, 2)))\n", " model.add(tf.keras.layers.GlobalAveragePooling2D())\n", " model.add(tf.keras.layers.Dense(NUM_CLASSES))\n", " return model\n", "\n", "model = get_model()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup Masterful\n", "\n", "The Masterful AutoML platform learns how to train your model by focusing on five core organizational principles in deep learning: architecture, data, optimization, regularization, and semi-supervision.\n", "\n", "**Architecture** is the structure of weights, biases, and activations that define a model. In this example, the architecture is defined by the model you created above.\n", "\n", "**Data** is the input used to train the model. In this example, you are using a labeled training dataset - [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html). More advanced usages of the Masterful AutoML platform can take into account unlabeled and synthetic data as well, using a variety of different techniques.\n", "\n", "**Optimization** means finding the best weights for a model and training data. Optimization is different from regularization because optimization does not consider generalization to unseen data. The central challenge of optimization is speed - find the best weights faster.\n", "\n", "**Regularization** means helping a model generalize to data it has not yet seen. Another way of saying this is that regularization is about fighting overfitting.\n", "\n", "**Semi-Supervision** is the process by which a model can be trained using both labeled and unlabeled data.\n", "\n", "The first step when using Masterful is to learn the optimal set of parameters for each of the five buckets above. You start by learning the architecture and data parameters of the model and training dataset. In the code below, you are telling Masterful that your model is performing a classification task (`masterful.enums.Task.CLASSIFICATION`) with 10 labels (`num_classes=NUM_CLASSES`), and that the input range of the image features going into your model are in the range [0,255] (`input_range=masterful.enums.ImageRange.ZERO_255`). Also, the model outputs logits rather than a softmax classification (`prediction_logits=True`).\n", "\n", "Furthermore, in the training dataset, you are providing dense labels (`sparse_labels=False`) rather than sparse labels.\n", "\n", "For more details on architecture and data parameters, see the API specifications for [ArchitectureParams](../api/api_architecture.rst#masterful.architecture.ArchitectureParams) and [DataParams](../api/api_data.rst#masterful.data.DataParams). " ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "tags": [ "outputPrepend" ] }, "outputs": [], "source": [ "model_params = masterful.architecture.learn_architecture_params(\n", " model=model,\n", " task=masterful.enums.Task.CLASSIFICATION,\n", " input_range=masterful.enums.ImageRange.CIFAR10_TORCH,\n", " num_classes=NUM_CLASSES,\n", " prediction_logits=True,\n", ")\n", "training_dataset_params = masterful.data.learn_data_params(\n", " dataset=training_dataset,\n", " task=masterful.enums.Task.CLASSIFICATION,\n", " image_range=masterful.enums.ImageRange.CIFAR10_TORCH,\n", " num_classes=NUM_CLASSES,\n", " sparse_labels=False,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next you learn the optimization parameters that will be used to train the model. Below, you use Masterful to learn the standard set of optimization parameters to train your model for a classification task.\n", "\n", "For more details on the optmization parameters, please see the [OptimizationParams](../api/api_optimization.rst#masterful.optimization.OptimizationParams) API specification." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "optimization_params = masterful.optimization.learn_optimization_params(\n", " model,\n", " model_params,\n", " training_dataset,\n", " training_dataset_params,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The regularization parameters used can have a dramatic impact on the final performance of your trained model. Learning these parameters can be a time-consuming and domain specific challenge. Masterful can speed up this process by learning these parameters for you. In general, this can be an expensive operation. A rough order of magnitude for learning these parameters is 2x the time it takes to train your model. However, this is still dramatically faster than manually finding these parameters yourself. In the example below, you will use one of the many sets of pre-learned regularization parameters that are shipped in the Masterful API. In most instances, you should learn these parameters directly using the [learn_regularization_params](../api/api_regularization.rst#masterful.regularization.learn_regularization_params) API.\n", "\n", "For more details on the regularization parameters, please see the [RegularizationParams](../api/api_regularization.rst#masterful.regularization.RegularizationParams) API specification." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# This is a set of parameters learned on CIFAR10 for\n", "# small sized models.\n", "regularization_params = masterful.regularization.parameters.CIFAR10_SMALL" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The final step before training is to learn the optimal set of semi-supervision parameters. For this Quickstart, we are not using any unlabeled or synthetic data as part of training, so most forms of semi-supervision will be disabled by default.\n", "\n", "For more details on the semi-supervision parameters, please see the [SemiSupervisedParams](../api/api_ssl.rst#masterful.ssl.SemiSupervisedParams) API specification." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "ssl_params = masterful.ssl.learn_ssl_params(\n", " training_dataset,\n", " training_dataset_params,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Training\n", "\n", "Now, you are ready to train your model using the Masterful AutoML platform. In the next cell, you will see the call to [masterful.training.train](../api/api_training.rst#masterful.training.train), which is the entry point to the meta-learning engine of the Masterful AutoML platform. Notice there is no need to batch your data (Masterful will find the optimal batch size for you). No need to shuffle your data (Masterful handles this for you). You don't even need to pass in a validation dataset (Masterful finds one for you). You hand Masterful a model and a dataset, and Masterful will figure the rest out for you." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MASTERFUL: Training model with semi-supervised learning disabled.\n", "MASTERFUL: Performing basic dataset analysis.\n", "MASTERFUL: Masterful will use 250 labeled examples as a validation set since no validation data was provided.\n", "MASTERFUL: Training model with:\n", "MASTERFUL: \t2250 labeled examples.\n", "MASTERFUL: \t250 validation examples.\n", "MASTERFUL: \t0 synthetic examples.\n", "MASTERFUL: \t0 unlabeled examples.\n", "MASTERFUL: Training model with learned parameters gorilla-honored-waltz in two phases.\n", "MASTERFUL: The first phase is supervised training with the learned policy.\n", "MASTERFUL: The second phase is semi-supervised training to boost performance.\n", "MASTERFUL: Warming up model for supervised training.\n", "MASTERFUL: \tWarming up batch norm statistics (this could take a few minutes).\n", "MASTERFUL: \tWarming up training for 515 steps.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "100%|████████████████████████████████████████████████████████████████████| 515/515 [00:51<00:00, 10.09steps/s]\n", "MASTERFUL: \tValidating batch norm statistics after warmup for stability (this could take a few minutes).\n", "MASTERFUL: Starting Phase 1: Supervised training until the validation loss stabilizes...\n", "Supervised Training: 100%|███████████████████████████████████████████████| 1659/1659 [01:39<00:00, 16.71steps/s]\n", "MASTERFUL: Semi-Supervised training disabled in parameters.\n", "MASTERFUL: Training complete in 2.6513574202855428 minutes.\n" ] } ], "source": [ "training_report = masterful.training.train(\n", " model, \n", " model_params, \n", " optimization_params,\n", " regularization_params,\n", " ssl_params,\n", " training_dataset,\n", " training_dataset_params,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The model you passed into [masterful.training.train](../api/api_training.rst#masterful.training.train) is now trained and updated in place, so you are able to evaluate it just like any other trained Keras model." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "model.evaluate(test_dataset.batch(optimization_params.batch_size))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "You have now finished training your first model with the Masterful AutoML platform!\n", "\n", "## Next Steps\n", "\n", "Now that you have finished this basic tutorial, we suggest exploring the rest of the documentation for more advanced examples and use cases. For example, you can learn how to use unlabeled data to further improve the performance of your model. Or you can learn about other vision tasks such as object detection and segmentation. Don't see a topic that you are interested in? Reach out to us directly on email at learn@masterfulai.com or join our Slack [commmunity](https://www.masterfulai.com/community). We are happy to help walk you through whatever challenge you are facing!" ] } ], "metadata": { "interpreter": { "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49" }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12" } }, "nbformat": 4, "nbformat_minor": 2 }