Pokémon Machine Learning

Can an A.I. distinguish Pokémon characters?

Machine learning takes many forms. Classifier functions are one type of artificial intelligence. We will train our own using Classify within the Wolfram Language.

Note: It is NOT recommended to simply “Evaluate Notebook” for this activity.

Pokémon Preface

As I was working on this lesson, I told my four- and six-year-old kids that I was working on a lesson with Pokémon. They were quite excited and insisted on seeing what I was doing. The part with images comes later, so I put this together to show them something.

Pokémon are entities within the Wolfram Language.

LinguisticAssistant

We can see all 911 Pokémon entities with EntityList or just randomly select a few.

RandomSample[EntityList[],5]

Recall that Through is a useful function when working with lists of Entities.

Through[{f,g,h}[x]]

We can use Through to show images of random Pokémon.

Through[RandomSample[EntityList[],5]["Image"]]

My kids immediately wanted to know their names. They don’t know much about Pokémon and pretty much all I know has been from poking around with these entities in the Wolfram Language.

Through[RandomSample[EntityList[],5]["Name"]]

But those names don’t match the pictures (probably) since they come from a different random sample. A quick pure function (# and &) column mapped /@ over the list gave me this.

Column[{#["Image"],#["Name"]}]&/@RandomSample[EntityList[],5]

My son can’t read yet and my daughter is learning, so they asked me to read the names to them. Of course, then they wanted to know all of the names.

namelist=["Name"]

I wasn’t up for reading all of that, but my computer is! Let’s start with a smaller list.

RandomSample[namelist,5]

Speak! (Recall that % refers to the previous output. )

Speak[%]

It’s reading the commas and quotes around each word. We can put it together as one string.

StringJoin[RandomSample[namelist,5]]

But we probably want spaces.

StringJoin[Riffle[RandomSample[namelist,5]," "]]

This probably sounds better.

Speak[%]

What about the whole thing? Well, while playing with this I learned that there are a few special characters in the list that cause Speak to stop. We can remove ♀ (Venus) and ♂ (Mars) and replace - (Hyphen) with - (keyboard dash). To type the hyphen type \ [Hyphen].

namestring=StringReplace[StringJoin[Riffle[namelist," "]],{"♀"->"","♂"->"","\[Hyphen]"->"-"}]

Before you Speak all that, beware that it takes about 11 minutes to read and I don’t know any way to stop the speaking. My kids lost interest before the end. My husband thought my computer caught some strange virus. I thought my computer sounded like artificial intelligence more than ever before. Remove the (* comments *) notation if you really want to hear it.

(* Speak[namestring] *)

Gathering and Organizing Data

One of the challenges with machine learning is accessing and organizing data for the computer to interpret. Fortunately, the Wolfram Language has many well-organized data sets that are readily accessible. Let’s start with Pokémon.

Among all of the "Pokemon" entities we can find those with a specific type. We will start with “Water” type. Also use EntityList to display the elements within the implicitly defined entity. Name this list so it is easier to refer to later.

waterlist=EntityList[Entity["Pokemon","Type"->"Water"]]

We can check the type of each Pokémon in the list using Through. This may take some time to evaluate the first time since it will download related values.

Through[waterlist["Type"]]

What do you notice about the type of each Pokémon? Some have more than one type. To simplify our data, let’s use ones that have only “Water” for type. For this we use Select with a pure function. Notice that we need the exact match for the result of “Type,” so copy/paste one of the “Water” entity properties from above. Name this list.

water=Select[waterlist,#["Type"]=={(Entity["Water", Entity["PokemonType","Water"], "Entity[\"PokemonType\", \"Water\"]", "Pokémon type"])}&]

Check that all of the Pokémon in the new list have only type “Water.”

Through[water["Type"]]

Let’s do the same to create a list of Pokémon of “Fire” type.

firelist=EntityList[Entity["Pokemon","Type"->"Fire"]]

Copy/paste one of the Pokémon from the resulting list and ask for “Type” to quickly get the “Fire” type entity property we need for the next step.

(Entity["Charmander", Entity["Pokemon","Pokedex0004:Charmander"], "Entity[\"Pokemon\", \"Pokedex0004:Charmander\"]", "Pokémon"])["Type"]

Copy/paste the “Fire” type entity property as needed below.

fire=Select[firelist,#["Type"]=={(Entity["Fire", Entity["PokemonType","Fire"], "Entity[\"PokemonType\", \"Fire\"]", "Pokémon type"])}&]

It appears there are fewer Pokémon with only type “Fire” than with only type “Water.” Let’s see how many we have since we will use some for training data and some for testing data.

Length[water]

Length[fire]

We are going to use the names of Pokémon as our first dataset.

Through[water["Name"]]

To train our classifier function, we will need to specify training data. These are examples that we give that have already been assigned a classification. Since we only have 32 “Fire” Pokémon, let’s take 20 randomly from the list to train for each type and save the rest for testing the quality of the classifier.

watertrain=RandomSample[Through[water["Name"]],20]

Complement will select the elements of our list that were not chosen for the training set. To test our classifier most effectively, we want the items in the test data to be different than the items in the training data.

watertest=Complement[Through[water["Name"]],watertrain]

There should be nothing in common among those two lists. One quick check is to use Intersection.

Intersection[watertrain,watertest]

We can also check the length of each list. Do you remember how many “Water” Pokémon we had in our list total?

Length[watertrain]

Length[watertest]

Now do the same thing with “Fire” type Pokémon. Create training data.

firetrain=RandomSample[Through[fire["Name"]],20]

Create testing data.

firetest=Complement[Through[fire["Name"]],firetrain]

Check the for overlap and the list lengths, if desired.

Intersection[firetrain,firetest]

Length[firetrain]

Length[firetest]

Training a Classifier Function

The Classify function in the Wolfram Language can take data in a few different forms. Since we have all of our examples already organized into two lists by category, it will be easiest to set up our data within Classify as an association. The <| and |> indicate an Association where each element is a Rule with a Key and a Value.

For our purposes, the keys will be the categories and the values will be the list of elements that go with each category. Be sure to use the training lists. Also, name this something so we can use it as a function more easily. I use “pnc” for “Pokémon Name Classifier.”

pnc=Classify[<|"Water"->watertrain,"Fire"->firetrain|>]

What type of Pokémon does it think I am? We can use the name of our function as a head just like “f” in f[x].

pnc["Abby"]

The classifier is an artificial intelligence that has been trained to learn what a “Water” type and  “Fire” type is based only on the names of Pokémon. No other properties are considered. It is purely based on the text and what the underlying neural network determines the probability to be for each category.

pnc["Abby","Probabilities"]

In my example, “Abby” is classified as “Fire” since its probability is higher (slightly higher) than for “Water.” Note that if different training examples are chosen or different options are used in Classify (or an updated version of Classify is published) then the results could be different. Neural networks trained on different data or with different parameters will “learn” different things.

Testing a Classifier Function

We can apply our classifier to all of our test examples. Let’s start with the “Water” type test data.

pnc[watertest]

If our classifier were perfect, what should that result look like? Let’s do the same for the “Fire” type test data.

pnc[firetest]

What do you think? How did our classifier do?

ClassifierInformation[pnc]

We can more formally test our classifier and get some analytics to consider. First we need to organize our testing data into one list with each example individually matched with its category. Thread will help with this.

Thread[{a,b,c}->x]

Start with “Water” test data.

Thread[watertest->"Water"]

We can do the same for “Fire” test data and we need to combine these results into a single list using Join.

testdata=Join[Thread[watertest->"Water"],Thread[firetest->"Fire"]]

ClassifierMeasurements will use the test data to check how well our classifier does with examples different than what is in the training data. “Accuracy” is one quick indicator.

ClassifierMeasurements[pnc,testdata,"Accuracy"]

We can also get more detailed results using a ConfusionMatrixPlot. Evaluate the following and see if you can figure out what each square means. Hover over each and read the tooltips to help.

ClassifierMeasurements[pnc,testdata,"ConfusionMatrixPlot"]

Why might some of them have been incorrectly classified? We can see the ones that were misclassified.

ClassifierMeasurements[pnc,testdata,"MisclassifiedExamples"]

What other analysis can we see?

ClassifierMeasurements[pnc,testdata,"Properties"]

Pokémon Image Classifier

Working with Pokémon names was a good warmup since it doesn’t take very long to download and work with text. However, images are much more fun.

Let’s look at our list of “Water” type Pokémon.

water

Copy/paste one of them as below and ask for its image.

(Entity["Squirtle", Entity["Pokemon","Pokedex0007:Squirtle"], "Entity[\"Pokemon\", \"Pokedex0007:Squirtle\"]", "Pokémon"])["Image"]

Let’s get all of the images for the “Water” type Pokémon. (This may take a little time to evaluate.)

waterimages=Through[water["Image"]]

Do the same for “Fire” type Pokémon.

fireimages=Through[fire["Image"]]

Now create the training set for “Water” type Pokémon.

waterimagetrain=RandomSample[waterimages,20]

And the testing data set for “Water” type Pokémon.

waterimagetest=Complement[waterimages,waterimagetrain]

A quick check of the number of elements in each list.

Length[waterimagetrain]

Length[waterimagetest]

The same for “Fire” type Pokémon. First, training set.

fireimagetrain=RandomSample[fireimages,20]

“Fire” image testing data.

fireimagetest=Complement[fireimages,fireimagetrain]

Quick check of list lengths.

Length[fireimagetrain]

Length[fireimagetest]

Now we are ready to train a new classifier function. I call this one “pic” for “Pokémon Image Classifier.”

pic=Classify[<|"Water"->waterimagetrain,"Fire"->fireimagetrain|>]

Choose a random image from the “Water” type test data.

RandomChoice[waterimagetest]

See what our classifier does with it.

pic[%,"Probabilities"]

Again, we can apply our classifier function to the entire list of images in our test data. Take a look at the results for “Water” type test data. How does our classifier do with this? If it were perfect, what should the list look like?

pic[waterimagetest]

Remember that none of the items in the test data are in the training data, so the classifier does not yet “know” the types of these examples. It has to use what it “learned” from the training data to figure them out.

pic[fireimagetest]

Keep in mind that “Water” and “Fire” don’t mean anything to the computer, whereas humans automatically make associations with each term and we get ideas about what images related to both may look like.

Imagine you were given a pile of images called “owuxf” and a pile of images called “zyrc” and told to create some rules for classifying future images. Since you know the category for each training image, this is an example of “supervised learning.” How would you approach this challenge?

RandomChoice[Alphabet[],4]//StringJoin

To use ClassifierMeasurments for our analytics, we need to organize our test data as we did earlier with names.

imagetestdata=Join[Thread[waterimagetest->"Water"],Thread[fireimagetest->"Fire"]]

Let’s see how accurate our classifier is with our test data. These evaluations will likely take longer than they did with the textual data of names.

ClassifierMeasurements[pic,imagetestdata,"Accuracy"]

We can also make a ConfusionMatrixPlot. Consider these results.

ClassifierMeasurements[pic,imagetestdata,"ConfusionMatrixPlot"]

Which ones did it get wrong? This will show the images along with their correct classifications.

ClassifierMeasurements[pic,imagetestdata,"MisclassifiedExamples"]

Some of these misclassified examples are understandable. Choose one of the images you think it should have classified correctly and copy/paste it to check its probabilities within our classifier. Was it a tough decision for the classifier? Or was it convinced in its classification?

pic[,"Probabilities"]

Imagery that seems obvious to us like flames or crab-like creatures is not necessarily obvious to the computer for classification. What could we do to make our classifier better? We could try training with a larger number of examples, but since there are a limited number of Pokémon images within our set we could only get so far and we would limit our number of testing examples.

Try training the classifier again using 25 training examples. Does it get better? How could we get more images for training and testing?

Another Example: Trees and Cars

We can get images directly from the internet. This doesn’t work too well for Pokémon, so let’s try something else.

Note that when you evaluate WebImageSearch, you may be asked to sign in with your Wolfram ID. Each search will use a few “Service Credits” that come automatically with your Wolfram ID. Evaluate $ServiceCreditsAvailable to see how many credits you have. See www.wolfram.com/service-credits for more information.

Don’t go crazy with numbers of images. We will stick with 30 so we can use 20 to train and 10 to test. First “tree” images.

trees=WebImageSearch["tree","Thumbnails","MaxItems"->30]

Now “car” images. Note that “Thumbnails” keeps the images smaller. With larger images, everything would be slower.

cars=WebImageSearch["car","Thumbnails","MaxItems"->30]

Below I put together the smaller code elements for organizing our data. This follows the same flow as what we did earlier. The semicolons ; will prevent us from seeing the output at each step. You could copy/paste and edit the pieces from above, or type it new. Think about what is happening at each step.

treestrain=RandomSample[trees,20];
treestest=Complement[trees,treestrain];
carstrain=RandomSample[cars,20];
carstest=Complement[cars,carstrain];
treescarstestdata=Join[Thread[treestest->"Tree"],Thread[carstest->"Car"]];

Now we will train a new classifier function for trees and cars.

treecarclassify=Classify[<|"Tree"->treestrain,"Car"->carstrain|>]

Let’s choose a random element from our tree testing set.

RandomChoice[treestest]

Try out the new classifier.

treecarclassify[%]

Choose a car from that testing set.

RandomChoice[carstest]

This time check how confident the decision is.

treecarclassify[%,"Probabilities"]

Interesting . . . What about all of the test data?

ClassifierMeasurements[treecarclassify,treescarstestdata,"Accuracy"]

And the ConfusionMatrixPlot?

ClassifierMeasurements[treecarclassify,treescarstestdata,"ConfusionMatrixPlot"]

Mine is perfect! What about yours? Does that make it more fun or less fun? Can you find a picture to put into the classifier to confuse it? Check probabilities when using the classifier to get an idea of how it is doing.

WebImageSearch["car in a tree","Thumbnails", "MaxItems"->10]

Another way to look at what is happening is with a FeatureSpacePlot. (Union combines the trees list and the cars list into one list.)

FeatureSpacePlot[Union[trees,cars]]

These images are so distinct that they are clearly grouped into two classes. How is this different than the Pokémon images?

FeatureSpacePlot[Union[waterimages,fireimages]]

If you look closely you should be able to see that images with similar features are close together. To help, hover with the mouse to see tooltips or reevaluate with options for increased ImageSize and LabelingSize.

FeatureSpacePlot[Union[waterimages,fireimages],ImageSize->600,LabelingSize->40]

FeatureSpacePlot is an example of “unsupervised learning.” Think of it as being given a pile of images with no labels nor categories and your task it to organize them all.

Option: Llama vs. Alpaca (More Automation)

I’m not talking hit points and attacks. Llama vs. Alpaca? Do you know the difference? Can we train the computer to figure it out for us?

As I put this together, I decided to rename the variables more generic “data1” and “data2” so it is easier to switch out the subjects of images retrieved. Also, note that I have it download 30 images, but only display five as a sample for visualization.

First, choose our items. These will also be the categories.

{item1,item2}={"llama","alpaca"}

Download images for the llamas.

data1=WebImageSearch[item1,"Thumbnails","MaxItems"->30];
RandomSample[data1,5]

Download images for the alpacas.

data2=WebImageSearch[item2,"Thumbnails","MaxItems"->30];
RandomSample[data2,5]

Just like for trees and cars, we can organize the data all at once.

data1train=RandomSample[data1,20];
data1test=Complement[data1,data1train];
data2train=RandomSample[data2,20];
data2test=Complement[data2,data2train];
alltestdata=Join[Thread[data1test->item1],Thread[data2test->item2]];

Train the classifier.

data12classify=Classify[<|item1->data1train,item2->data2train|>]

Check the ConfusionMatrixPlot

ClassifierMeasurements[data12classify,alltestdata,"ConfusionMatrixPlot"]

Do you want to see the misclassified examples?

ClassifierMeasurements[data12classify,alltestdata,"MisclassifiedExamples"]

If  you have time, try downloading 100 images for each and training with 80 examples instead of 20. Does this improve your classifier? What about dolphins and porpoises?

Option: More Categories

We can certainly add more categories. This will increase training time, but could make for more interesting functions.

{item1,item2,item3}={"horse","donkey","mule"}

Download images for horses.

data1=WebImageSearch[item1,"Thumbnails","MaxItems"->30];
RandomSample[data1,5]

Download images for donkeys.

data2=WebImageSearch[item2,"Thumbnails","MaxItems"->30];
RandomSample[data2,5]

Download images for mules.

data3=WebImageSearch[item3,"Thumbnails","MaxItems"->30];
RandomSample[data3,5]

Add additional categories to our data processing.

data1train=RandomSample[data1,20];
data1test=Complement[data1,data1train];
data2train=RandomSample[data2,20];
data2test=Complement[data2,data2train];
data3train=RandomSample[data3,20];
data3test=Complement[data3,data3train];
alltestdata=Join[Thread[data1test->item1],Thread[data2test->item2],Thread[data3test->item3]];

Train the classifier.

data123classify=Classify[<|item1->data1train,item2->data2train,item3->data3train|>]

Now take a look at the ConfusionMatrixPlot.

ClassifierMeasurements[data123classify,alltestdata,"ConfusionMatrixPlot"]

We can see the misclassified examples, but I cannot find an easy way to have it show the examples along with their incorrect classification.

ClassifierMeasurements[data123classify,alltestdata,"MisclassifiedExamples"]

Aside: What are all Pokémon Types?

If we wanted an example with even more categories, we could do this with more types of Pokémon. What are all of the Pokémon types?

Sort[Union[Flatten[LinguisticAssistant["Type"]]]]

Epilog: “Mom, can you show me all of the pink Pokémon?”

As I continued working on this and my kids were peeking at the various images I had on the screen, my daughter asked, “Mom, can you show me all of the pink Pokémon?” This led to a good computational thinking challenge. How can I get the computer to pull out just the pink ones? Or ones mostly pink?

First, I downloaded all of the Pokémon images. This takes some time the first time, so it is (* commented *) below. If you want to download a notebook that already has them use this link: www.abbymath.com/CT/Activities/Pokemon-All-Images.nb.

(* pkimages=LinguisticAssistant["Image"] *)

My idea was to reduce each image to just one color and then find the ones that are nearest to pink. DominantColors was my function of choice to reduce images to one color, but the function seemed to take into account the background as black or white for most images. DominantColors has a Masking option where you can ignore regions of the image. To make the mask for the background, we can simply Binarize the image. However, as of writing this, of the 911 Pokémon images, the last 142 of them have only 3 image channels rather than 4, so the Binarize function works opposite on them. I included an If statement to fix this part. . . and submitted a bug report (no pun intended). This also takes time to evaluate.

Note: The issue with image channels could also affect the training and testing of the classifier functions above.

(* pkcolorlist=Flatten[DominantColors[#,1,Masking->If[ImageChannels[#]==3,ColorNegate@Binarize[#],Binarize[#]]]&/@pkimages] *)

If you didn’t evaluate that, you can still open the example below to see what it looks like. If you copy/paste the list into a new cell and name it “pkcolorlist” the following code will work.

So now I will select ones that are near to pink. ColorDistance is perfect for this.

Select[pkcolorlist,(ColorDistance[Pink,#]<0.2)&]

However, those don’t look quite like the shade of pink I was going for.

Lighter[Pink,0.6]

Select[pkcolorlist,(ColorDistance[Lighter[Pink,0.6],#]<0.2)&]

Interesting, maybe a tighter range would help.

Select[pkcolorlist,(ColorDistance[Lighter[Pink,0.6],#]<0.125)&]

I need now to associate the images with the colors so we can see the Pokémon, and I know she will want to know the name of each one too. I put together the ordered triples of name, image, and dominant color for all of the Pokémon.

pkdata=Transpose[{namelist,pkimages,pkcolorlist}]

Now, as above, we can select the ones with the dominant color (the third element of each triple) close to pink.

Select[pkdata,(ColorDistance[Lighter[Pink,0.6],#[[3]]]<0.125)&]

And I’ll add a little formatting to that.

Grid[Partition[Column/@Select[pkdata,(ColorDistance[Lighter[Pink,0.6],#[[3]]]<0.125)&],UpTo[5]],Dividers->All]

Open the example below for the result, if you don’t want to evaluate everything.

I don’t know whether that is “all” of the pink ones, but it certainly satisfied my daughter. Of course, my son now asks to see the yellow ones because he wants Pikachu (the only Pokémon name he knows) to appear. Luckily, with decent programming, I should only need to make one switch.

Grid[Partition[Column/@Select[pkdata,(ColorDistance[Yellow,#[[3]]]<0.125)&],UpTo[5]],Dividers->All]

Yellow isn’t quite the same as Pokémon yellow. What yellow is Pikachu?

DominantColors[LinguisticAssistant]

So I’ll use that yellow for the color. (Notice the dominant black. That’s why I needed the Masking option earlier.)

Grid[Partition[Column/@Select[pkdata,(ColorDistance[,#[[3]]]<0.125)&],UpTo[5]],Dividers->All]

“Now can you show me all of the Pokémon that look like unicorns?”

Abby Brown
www.abbymath.com
CT@TP: Computational Thinking at Torrey Pines High School
November 2018