CycleGAN and Chef-RNN update

There has been a development in image generation that I find absolutely fascinating. It’s based off Generative Adversarial Networks, which are a very powerful and promising model for image generation. A recent modification to this architecture, the Conditional GAN has been making some waves, as it allows for the translation of one type of image to another, but it has the drawback of requiring databases of before and after images with roughly 1:1 correspondence, which is difficult to find, dramatically limiting the applications. However, very recently, the folks at UC Berkley made a few additional modifications which remove this requirement, creating the CycleGAN architecture. I’ve downloaded the source and been using the Flickr api to do some experiments with it, including trees <=> flowers, summer <=> fall, summer <=> winter, and landscape <=> desert. Legal disclaimer: I don’t own the rights to any of the images here. Unfortunately I neglected to get the photographers’ info when I scraped the images, but if anyone knows the creator of any of the original images please let me know. Here’s a few example images I’ve gotten so far:

Trees<=>flowers doesn’t work very well, which isn’t too surprising, but it is pretty entertaining sometimes what it does. It found pretty early on that it can do decently by just inverting the colors, but eventually the behavior got more complex and started making gross brown flowers:

summer<=>fall works just… absurdly well. It’s a bit scary, and some of the results are really pretty. With some parameter tuning and more (and better sanitized) data, this could be really cool! I’m definitely going to do some more experimentation with this.

This slideshow requires JavaScript.

summer<=>winter also works, though I couldn’t get it looking as good as the authors of the paper did. These examples are a bit cherry-picked, though– it never really learned how to fully get rid of snow, but it’s really good at color balance adjustments that make it feel way more wintery/summery.

This slideshow requires JavaScript.

The “desertifier” was largely unsuccessful. It never learned how to make things into sand like I’d hoped, but I didn’t train it for nearly as long as the others, and the success cases give me hope that it could learn:

This slideshow requires JavaScript.

 

Essentially what I’ve found is that the network doesn’t like to totally get rid of anything or hallucinate new things, even when it would make sense to do so. For example, if it gets rid of some water to make a desert, it might not be able to put it back- because of how the Cycle-GAN works, it needs to be able to reconstruct the original image. What it is really good at is changing colors and patterns, less good at structural stuff. I’d bet that you might be able to improve this behavior with skip connections between the initial transformation and the reconstruction pass. This would be similar to the “u-net” encoder-decoder architecture described, except the connections between the encoder and the decoder would also connect the first step of the titular cycle to the second. It might defeat the purpose a bit, but as long as there’s still an information bottleneck it might help.

Robo-Chef strikes again (barbecue sauce edition)

Finally, I discovered that I never did any experimentation with the sequence length I used to train my robo-chef, which would put a hard limit on how long it could remember things. Here are some recipies from a network trained with a much longer memory (3.5X longer)

Low Temperature (0.3)

title: corn soup
categories: soups
yield: 1 servings

1 c buttermilk
1 c chicken stock
1/2 c butter
1 c sour cream
1 ts salt
1/8 ts pepper
1 ts paprika
1 c flour
1 ts baking powder
1 ts sugar
1/2 ts baking soda
1/2 ts cayenne pepper
1 tb butter
1 c cheddar cheese, shredded

combine cornmeal, flour, salt and pepper. stir in buttermilk and
corn flakes. stir in cheese and cheese. bake in a greased 9-inch
square pan in a 350 f oven for 20 minutes. serve with sauce. source:
pennsylvania dutch cook book – fine old recipes, culinary arts press,
1936.

That ain’t a soup! But look, it remembered the buttermilk! If you follow the instructions, you might end up with… some kind of cheesy corn-flake pastry? Weird.

title: cranberry cream cheese cake
categories: cakes, cheesecakes
yield: 1 cake

1 pk cream cheese, softened
1 c sugar
2 eggs
1 ts vanilla
1 c flour
1 ts baking powder
1/2 ts salt
1 c chopped walnuts
1 c chopped nuts
1 c chopped nuts

heat oven to 350 degrees. combine crumbs, sugar and cinnamon in a
small bowl. add egg and mix well. spread over crust. bake at 350
degrees for 10 minutes. cool. cut into squares. makes 12 servings.

source: canadian living magazine, apr 95 presented in article by diana
rosenberg

That’s a lot of nuts! Also it forgot the cranberries. Once again, the instructions seem to be totally independent of the ingredients, but would make… some kind of cinnamon crumb pie? That actually sounds kind of delicious. I bet you could probably make something really tasty out of this one.

title: bran muffins (cookbook style)
categories: breads, fruits
yield: 12 servings

1 c flour
1 ts salt
1 ts baking powder
1/2 ts salt
1 c sugar
1 c milk
1 c milk
1 egg
1/2 c milk
1/2 c buttermilk
1 ts vanilla
1 c chopped pecans

combine flour, baking powder, salt and sugar. cut in shortening with
pastry blender until mixture resembles coarse meal. add egg and mix to
blend. stir in buttermilk and egg mixture. stir in raisins and nuts.
pour into greased 9-inch square baking pan. bake at 350 degrees for
30 minutes. cool in pan on rack 10 minutes. remove from pans and cool
completely. store in airtight containers. makes 12 servings. source:
pennsylvania dutch cook book – fine old recipes, culinary arts press,
1936.

Wow! Not a bran muffin but this sounds… really coherent. This is one of the few times I think the instructions might result in a somewhat reasonable food.

BUT WAIT, are you ready for BARBECUE SAUCE?

title: crunchy barbecue sauce
categories: bbq sauces
yield: 1 servings

1/2 c brown sugar
1/4 c worcestershire sauce
2 tb brown sugar
1 tb cornstarch
1 tb water
1 ts cornstarch
1/2 ts salt
1 tb cornstarch
1 tb water
1 tb soy sauce
1 ts sesame oil
1 tb cornstarch
1 ts sugar
1 ts soy sauce
1 ts sesame oil

combine all ingredients except salt and pepper in a large saucepan.
bring to a boil, reduce heat and simmer for 1 hour. add cornstarch
and cook until thickened. stir in chicken and cook until thickened.
serve over rice.

And  here we go. Crunchy Barbecue Sauce. That full 1/4 cup of worcestershire sauce. The tons and tons of cornstarch. Oh. Man. What is going on? Weirdly enough, the instructions seem spot-on (except the last two sentences get a bit weird). Here’s a condensed list of the ingredients just to see if it makes sense:

2/3 c brown sugar
1/4 c worcestershire sauce
4 tb cornstarch
2 tb water
1/2 ts salt
2 tb soy sauce
2 ts sesame oil
1 ts sugar

The amounts might need adjusting, but at least it has the right flavors going on. I guess the amount of sugar is why it’s crunchy. But wait! There’s more! Barbecue sauces #1, #2, and #1 [sic]:

title: barbecue sauce #2
categories: sauces
yield: 1 servings

1 c chicken broth
1 c chicken broth
1 tb cornstarch
1 tb soy sauce
1 ts sugar
1 ts sesame oil
1 ts sesame oil
1 ts sesame oil
1 ts sesame oil
1 ts sugar
1/4 ts salt
1 ts sesame oil
1 ts sesame oil
1 tb sesame oil
1 tb cornstarch
1 ts sesame oil
1 ts sesame oil
1 ts sesame oil
1 ts cornstarch
1 ts sesame oil
1 ts sesame oil
1 ts sesame oil
1 ts sugar
1 ts sesame oil
1 ts sesame oil
1 ts sesame oil
1 tb cornstarch
1 tb sesame oil
1 tb water
1 tb sesame oil
1 tb chinese chili paste

cut the chicken into small pieces. heat the oil in a large skillet over
medium heat. add the chicken and cook, turning the chicken frequently,
until the chicken is cooked through, about 5 minutes. remove the chicken
to a plate. remove the chicken from the skillet and keep warm. add the
chicken to the pan and stir-fry for 1 minute. add the chicken and
stir-fry for 1 minute. add the chicken and cook, stirring, for 1
minute. add the chicken and continue to cook for another 2 minutes.
return the chicken to the pan. add the chicken and stir-fry for 1
minute. add the chicken broth and cook for 1 minute. add the chicken
and stir-fry for 2 minutes, then add the sesame oil and stir until
combined. add the chicken and stir-fry for 1 minute. add the chicken
broth and cook, stirring constantly, until the sauce thickens. stir in
the chicken broth and cook for another 2 minutes. stir in the cornstarch
mixture and cook for 1 minute. stir in the chicken broth and cook,
stirring, until the sauce thickens. serve immediately.

serves 4.

from the files of al rice, north pole alaska. feb 1994

Did you remember to cook the chicken? How about the sesame oil?  Ok, good. What about the chicken? Also this recipe is from “north pole, Alaska.”

title: barbecue sauce #1
categories: sauces
yield: 1 servings

1 c water
2 tb soy sauce
1 tb brown sugar
1 tb cornstarch
1 tb soy sauce
1 tb sugar
1 tb cornstarch
1 tb water
1 tb cornstarch
1 tb water
1 tb cornstarch
1 tb water

combine all ingredients in a large saucepan. bring to a boil, reduce heat
and simmer, uncovered, for 1 hour, stirring occasionally. stir in
cornstarch mixture and cook 3 minutes more. stir in cornstarch mixture
until smooth. add salt and pepper to taste. serve over rice.

Yeah, ok, that one makes more sense. Actually that seems to be a legit barbecue sauce. Cool! Unfortunately it’s also the most boring barbecue sauce ever because there are only actually four ingredients and one of them is water.

title: barbecue sauce #2
categories: sauces
yield: 1 servings

1 c brown sugar
1 c water
2 tb soy sauce
1 tb sugar
1 ts salt
1 ts salt
1 ts chili powder
1 ts ground cinnamon
1 ts ground coriander
1 ts ground cardamom
1 ts ground cumin
1 ts ground coriander
1 ts ground ginger
1 ts ground coriander
1 ts ground cardamom
1 ts ground cardamom
1 ts ground coriander
1 ts ground cinnamon
1/4 ts ground cloves
1/4 ts ground cloves
1/4 ts ground cardamom
1/4 ts ground cloves
1/4 ts ground cloves
1/2 ts ground cumin
1/4 ts ground cloves
1/2 ts ground coriander
1/2 ts ground cloves
1/2 ts ground cloves
1/2 ts ground coriander
1/2 ts ground cardamom
1/4 ts ground cloves
1/4 ts ground cumin
1/4 ts ground coriander
1/4 ts ground cumin
1/2 ts ground cardamom
1/2 ts ground cardamom
1/4 ts ground cloves
1/4 ts ground cardamom
1/4 c chopped fresh parsley

combine all ingredients in a large saucepan. bring to a boil, reduce
heat and simmer for 15 minutes. add chicken and cook 10 minutes more.
serve over rice or noodles.

recipe by : recipes from the cooking of indian dishes by cathy star
(c) 1994. typed for you by karen mintzias

Oh jeez what happened. This one starts off well and then SPICES. At least it’s more interesting than the last one!

Okay let’s turn up the heat and see what happens.

Reasonable temperature (0.55)

title: barbecue sauce for steaks
categories: sauces, marinades, low-cal
yield: 1 servings

1 c burgundy wine
1 tb soy sauce
1 c water
1 tb vegetable oil
1 garlic clove, crushed
1 md onion, chopped
1 garlic clove, minced
4 chicken breasts, boned,
-skinned, cut into 1/2-inch
-strips
1 tb cornstarch
1 tb water
1 tb cornstarch paste
1 tb cold water

mein sauce: combine the marinade ingredients and set aside.

in a large pot, bring the water to a boil. add the garlic and stir
for 2 minutes. add the spices and reduce the heat. simmer, covered,
for 10 minutes. remove the pan from the heat and store in an
airtight container.

makes about 1 cup.

Hahaha, another barbecue sauce! Nein, mein sauce!  Too bad the instructions are basically boiled garlic. At this point we’re done with the barbecue sauce (alas).

title: grilled portabello mushrooms with cashews
categories: mexican, vegetables, indian
yield: 4 servings

1 tb mustard seeds
1 tb salt
1 tb ground ginger
2 tb peppercorns
1 tb ground fenugreek
1 tb ground cardamom
1 tb ground cardamom
1/4 ts ground cardamom
1 ts ground cardamom
1 ts ground cumin seeds
1 tb chili flakes
1 ts chili powder
1 ts ground tumeric
1 ts ground cumin

mix all the ingredients together and serve cold.

Oh god this one made me laugh so hard. It’s so beautifully simple! I hope you like spices, because that’s what’s for dinner. Anyone know if this would make any sense at all as a spice mix?

title: hot-pepper caribbean black bean sauce
categories: sauces, vegetables
yield: 1 servings

1 c chicken broth
1 c cooked rice
2 tb soy sauce
1 tb sugar
2 tb soy sauce
1 tb sesame oil
1 c chicken broth
1 ts sugar
1 ts salt
1/4 c soy sauce
2 tb cornstarch mixed with 1/4
-cup water, as needed

serves 4.

place the rice in a saucepan and bring to a boil. add the chicken broth
and cook under medium heat for 5 minutes. remove the chicken from
the pot. add the chicken and cook for another 10 minutes. remove
the chicken from the pan and set aside.

add the chicken and the remaining ingredients and simmer 15 minutes.
skim off the excess fat and return the chicken to the pot.

serve the chicken and sauce with the chicken and a salad.

So wait, what am I supposed to do with the chicken again?

Ok, now let’s really get cookin’!

High Temperature (0.8)

title: junk-joint salet burger
categories: pasta, pork
yield: 6 servings

stephen ceideburg
1 c spam luncheon meat
1 bag fully cooked bacon
2 c sliced green onions
1 stalk fresh basil – chopped
1 clove garlic — sliced
1 c sauerkraut
1/3 c sugar
4 ts lemon-soy sauce
1 ts hot sauce
1/2 ts salt
1/2 c water

in large skillet over medium heat, heat oil over medium heat; cook
garlic until soft, but not browned, about 15 minutes. add remaining
ingredients except noodles; cook for 2 to 3 minutes or until thickened,
stirring after 5 minutes. stir in raisins and simmer for 1 minute.
serve over chicken.

source: taste of home mag, june 1996

It’s a what? I guess if you go to a Junk-Joint and order a Salet Burger this is what you get. Also those ingredients… substitute ground hamburger for spam and you might be able to make a decent, but weird, burger. What’s really neat is that the instructions remember that this is supposed to be pasta (which the ingredients conveniently forgot).

title: chicken & milk grits
categories: poultry
yield: 2 servings

1 whole chicken
– skinned, fat from fresh
– chicken breast
– cut into 1/2 inch strips
1/4 c low-fat cottage cheese
1 (10-oz.) can whole tomatoes
— drained, chilled and
– drained
1 tb olive oil
2 tb sour cream
salt and pepper
4 flour tortillas
cooked spaghetti
mushrooms
parsley
sauted mushrooms

combine flour, salt & pepper in a medium bowl. cut in margarine
until particles and can leave from tip. pat the mixture into a baking dish
and sprinkle with the cheese. bake, basting every 15 minutes, until
crust is golden brown, turning the cheese over after 35 minutes.
meanwhile, mix the egg and water in a small bowl. stir in the remaining
ingredients. pour grated cheese into the pie shell and bake for
20 minutes. remove from oven. sprinkle with roquefort on top.

What… what is this? The name is weird, the ingredients are… confusing (tortillas and spagetti?) and the instructions are for… some weird cheese pie. I don’t really know what to make of this. I think I may need to call a chef to reconcile some of these recipes for me. That said, if you did manage to actually make this it might not be half bad if you made some pretty liberal substitutions and improvisations.

But wait, are you ready to

title: do the cookies
categories: cookies, breads
yield: 1 servings

1 c sugar
1 c shortening
1 tb baking powder
1 c buttermilk
1 ts soda
2 eggs
1 ts vanilla
1/2 ts almond extract
cinnamon

————————–filling——————————-
4 c confectioners sugar
2/3 c water
sprinkles
chocolate chips, karola or
crystallized chocolate
-red berries, for garnish

preheat oven to 350. mix cake mix, salt, flour and salt. beat egg
whites with salt until foamy. gradually add remaining 1/2 cup sugar,
beating until stiff. beat in vanilla extract and vanilla and fold into
batter. pour into remaining tins. bake in a 350 degree oven for 30
minutes until done. cool 1 hour before removing from pan. per
serving: 101 calories, 1 g protein, 12 g carbohydrate, 3 g fat, 3 g
carbohydrate, 0 mg cholesterol, 30 mg sodium.

note: if simple way to do not mix the frosting together with egg and
sugar and your dough will hold the mixture in the freezer.

DO THE COOKIES? Oh man. This is a full, internally consistent, somewhat logical cookie recipe. And, AND, it has nutrition facts. So you know it’s healthy! Wow. Do the cookies!

title: anglesey’s french prepared chicken wings
categories: italian, poultry
yield: 6 servings

1 chicken breast meat, cut into
-serving pcs.
1 c cooked rice
1 c vinegar
1/2 c vinegar
4 cloves garlic, minced
1 tb cider vinegar
1 tb dijon mustard
2 tb red wine vinegar
4 tomatoes, chopped
1 tb celery, fresh, snipped
1 red pepper, julienned,
-seeded and diced
1 tb parsley, chopped
1/4 c red wine vinegar
salt
freshly ground black pepper

thaw and drain chicken (roll up the sides of the chicken). trim and
cut the chicken into strips. combine the chicken with the pork mixture
with the salt, pepper and thyme. mix everything together gently and
add to the chicken mixture. cover and refrigerate for at least 4 hours
or overnight.

…and then what? Wait, do you serve this raw? It put SO MUCH WORK into those ingredients (look at all that vinegar) and then forgot to actually cook the meat (arguably the most important step).

title: grilled portobello mushrooms
categories: vegetables
yield: 4 servings

1 md garlic clove, crushed
1 sm onion, chopped
2 tb butter
8 oz fresh plantains, leaves,
-frozen
– thawed
1/4 c raisins
1 tb balsamic vinegar
1 tb worcestershire sauce
2 tb tamarind sauce
1/8 ts pepper
1 ts lemon juice
1/2 ts garlic, minced
salt to taste
freshly ground pepper

put first 4 ingredients in a bowl, mix well and stir into corn mixture.
in a 2-quart saucepan, heat the butter and 2 tablespoons of the
frankfurter seasoning and add the cooked rice and stir until the sauce
thickens and serves 4 to 6. makes about 2 1/2 cups

recipe by : cooking live show #cl8726

This one is actually so close. If only it actually included portobello mushrooms! Also, I want to emphasize: 8 ounces of fresh plantain leaves, frozen, then thawed. WHAT.

title: chocolate sunchol apple cake
categories: cakes, chocolate, vegetables
yield: 1 servings

1 c brown sugar packed
2/3 c butter or margarine
1/3 c cocoa
2 ts vanilla
1/2 ts almond extract
1 c chopped nuts
1/2 c coconut
3/4 c sour amount of cold water
1/2 c flour
1/2 ts baking soda
1/4 ts salt

in a large bowl, cream margarine. add sugar, flour, vanilla, and eggs.
mix thoroughly. pour into prepared pan. bake 45 minutes or until
oblong starts to pull away from sides of pan and a wooden pick inserted
into center comes out clean. cool in pan on wire rack for 5 minutes.
remove cake from pans to wire rack. remove from cookie sheet to wire
rack and cool.

Another legit pastry… thing! Also with some confusing oddities.

title: home-made potato salad
categories: salads, greek, vegetables, pork
yield: 6 servings

1 lg ripe avocado,cubed
1/2 ts lemon juice
1 c chicken broth
1/2 ts ground cumin
1/4 ts salt
1/8 ts pepper
1/4 ts cumin
1 c cooked rice
1/4 c water
1 c broccoli florets
3 c chicken broth
1/4 c parsley, finely chopped
2 tb green onions, finely chopped
pinch nutmeg

break up cooked peas. saut� garlic in oil until soft. stir in flour
and stir until smooth. combine all ingredients. cook and stir over medium
heat until sauce boils and thickens. cool
1/4 hour before serving.

That’s a weird potato salad. But you know what, it might be good! Dang, I’m getting hungry.

I also had some fun turning the temperature up REALLY high. The recipes get… out there. Have a look

Crazy high temperature (1.1)

  title: horey dipping sauce-(among-burlas)
categories: japanese, sauces, chutneys
yield: 1 cup

1 c cream and port:
1/8 c whole-wheat flour
2 tb canola; grated
32 oz tomate; sliced
3 tb crumbled honey
3/4 c onions; chopped fine
1/4 c bell pepper; chopped
2 ts accent
1 c chicken; cooked,chopped
1/4 ts poultry seasoning; if desired
1 1/2 ts peanut oil
4 green onions; peeled and
-thinly sliced

saute onion, green onions and celery just until onions begin to soften.
place for 1 minute add beans, water, chicken bouillon cube, green
onions, and cabbage. bring to a boil. boil uncovered quickly,
covered, for 20 minutes.

ladle into warm soup bowls. top with chicken and serve.

note: you can substitute cooked vegetables for thin strands of
rice. potatoes can go to be with really better some commercially back
in place of this. i also added this in backbone. wonderful!

Yeah ok, some kind of… honey peanut canola chicken thing. I can live with that. Let’s get weirder!

title: crispy ripe fetureurs
categories: chinese, game
yield: 8 servings

8 ea fresh artichokes; each 3″ died
2 ea garlic cloves; minced
1 ea onion; grated, or marinade
12 oz goonne, whole black bean; **
1 ea bay leaf
1/2 ts ginger; fresh, peeled,
-ground
1/2 ts salt
3 tb chili berry or vinaigrette
2 tb soy sauce
3 tb vinegar
1 tb dijon mustard
salt and freshly ground
-black pepper
1 lb peeled carrots
3 tb peanut oil
4 1/2 c water
1 1/2 lb chicken; quartere

chop all of this liquids into separate bowls. put 2 mayonnaise into a
large bowl. stir in the garbanzo beans until pureed. add the
pork and mix thoroughly. toss the spatula and toss well with the
first mixture. set aside for a few hours before coating.

remove the skin to a dinnworm enough to act a lasagle, starting in the
rosette. roast, uncovered, in a hot 350 f. oven for 15 minutes.
meanwhile, wash the lettuce, well, tuver peel the green palm. hold the
carrots very finely. but do not rinse them. after they are cooked
to the texture, place the sauce in another hot skillet largeroune,
and add enough hot water to cover it.

cover and simmer the soup until the rice is done, about 4-6 hours,
date to see dowel up. pour into hot sterilized jars to make sure your
amber liquid has reheated. chiln quickly if the barbeque side is
chilled and stored in a storage tin, loosely probably one day, watch
until chiles are soupy, thoughly barbecued, about 37 hours, or in the
refrigerator to marinate the meat or your beurre but may be made up to 2
days, covered.

cornstarch mixture: this sauce manie sirfully begin to should
be approximately 3 cups of cooking your toothpicks.

from black beans & the ultimated cookery.. conf: grasne ago
vyra bennett un, tradhlenl

Oh jeez too weird too weird. There’s so much going on in here. What is a dinworm? For that matter, What’s a lasagle? The first paragraph is 100% gold. Also, this recipe takes a long time. First, you have to stir some beans until pureed. You have to puree beans with a spoon. Then you cook the… dinnworm… for 15 minutes, then simmer the soup for 4-6 hours, then watch the “chiles” until they are “soupy, thoughly barbecued,” which takes 37 hours or up to 2 days.

title: fried parmesan 5-mint
categories: appetizers
yield: 8 servings

4 fresh ham, thinly sliced as
-slices 1/2 inch thick
3 md fresh mushrooms, thinly
sliced
1/4 c toasted sliced green onions
– (finely)
2 lg black pepper, the
1 tb grated parmesan or sandwich
1/4 c lowfat yogurt
1/3 c sour cream
lime wedges
parsley sprigs

1. place remaining ingredients in each of a bl. plate, cover and microwave
on 300of until cheese melts (about 15 seconds). serve at once, with
salsa.

Somehow the network made an OK sounding chip dip. I love that the instructions are basically “throw everything in a bowl and microwave.”

In summary, holy cow! This is so much more coherent than my previous experiments, and with only one night of training! I definitely need to dig into this a bit more. If you’ve somehow made it to the end of this post and want MORE, I’ve found another blog that does similar things. Check it out!

Advertisements
Posted in char-rnn, CycleGAN | Tagged , , , , , , | 2 Comments

Double Jeopardy

Because I was gone for most of the weekend, (and because I didn’t want that awful jokebot being the first thing people see of this blog) I retrained the jeopardy network to some amusing results. Take a gander:

THE SPORTING LIFE,$400,’The first of these in the U.S. was the first to control the state of Maryland’,a statue of Martin Luther King, Jr.

THE OLD WEST,$800,’This country is the only one of the world’s largest countries’,Chile

THE BIBLE,$400,’This composer of the 1999 film The Sound of Music was a star of the 1995 film The Sound of Music’,John Steinbeck

THE SOUTHERN DANDY,$400,’The name of this country is a synonym for a state of a country’,South Africa

John Steinbeck composed and starred in a 1999 remake of The Sound of Music, apparently. Also a statue of MLK Jr. took over Maryland, and Chile is bigger than I previously thought. The more you know! Increasing the temperature a bit:

A WORLD OF BEER,$400,’The first of these in the U.S. was a company in 1999′,a balloon

THE CARIBBEAN,$400,’The sea is the capital of this country’,Chile

THE FIFTH,$400,’The name of this body part is from the Latin for to strain into a string’,a contract

THE STORY OF O,$400,’This country is the second largest city in the world’,Martinique

BEAR SCREEN,$200,’This author of The Secret Garden was based on a 1989 film about a stripper who was a little boy’,James Bond

FICTIONAL DETECTIVES,$1000,’This 1954 film is set in the 1997 film seen here’,The Man Who Shot The Rainier

ART & ARTISTS,$1000,’This Southern country was a colony of the New York City Company in 1968 & is now a capital city’,Berkeley

The network continues to fail at geography. In addition to being the largest country in the world, the capital of Chile is the sea. Also, some of Jame’s Bond’s sordid origins and a 1950’s sci-fi detective film, The Man Who Shot The Rainier, which I kind of want to watch. Stepping up the temperature some more, we learn about American history:

THE CIVIL WAR,$600,’In 1990 the Confederacy allowed this country to the U.S. Constitution’,South Africa

THE 1980s,$400,’This son of a president was a senator from 1948 to 1972′,James Buchanan

THE CIVIL WAR,$800,’This secretary of the American Idol was buried in the first festival of the State Department’,Harry Truman

BOOKS OF THE ’60s,$400,’This Seattle children’s team was based on a 1986 movie based on a series of books’,Stevie Wonder

COLLEGES & UNIVERSITIES,$1000,’On April 1, 1998 this country became the first black president to control the U.S. Army’,Japan

BIBLICAL PEOPLE,$400,’This man who resigned as a lawyer in 1994 was the first female president of the Confederacy’,John Adams

THE 1980s,$300,’This man who died in 1978 was a president of the Senate from 1934 to 1990′,Benjamin Franklin

THE NEW YORK TIMES TECH BIZ,$300,’This country’s 1969 exploits were completed in 1936′,Australia

STATE CAPITALS,$1000,’This capital city was founded in 1939 by the El Capitan of New York City’,Columbus

Okay, I snuck some Australian time travel in there. I also got this absolute gem:

THE FIFTH,$400,’This president was the first president to serve as president’,John F. Kennedy

Let’s keep this going:

WORLD CAPITALS,$1000,’The company that contains the largest island in the U.S.’,Canada

THE END,$1000,’It’s the body of water in the Confederacy that shares its name with a former capital’,Barcelona

STATE CAPITALS,$200,’The name of this capital city is a 2-word name for a pope’,Beijing

THE ASTRONAUT HALL OF FAME,$200,’In 1969 he was called the last world championship to win the major series title’,Alexander Hamilton

FAMOUS COLLEGE DROPOUTS,$200,’In 1998 this president was a commander of the Confederate Army’,Adolf Hitler

THE OLD WEST,$200,’In 1991 this American became the first woman to be a consul on the Moon’,Britney Spears

THE SOUTH PACIFIC,$400,’It’s the only country that makes it to the Atlantic Ocean’,Australia

WOMEN AUTHORS,$400,’In 1987 this TV heroine was a spin for the No. 1 hit Heart of Darkness’,John Paul Jones

THE ELEMENTS,$400,’This compound is a major work of the subatomic particle that makes surreal & trick’,a sodict

Godwin’s law invoked! Also, it invented a subatomic particle. Canada is a US-based company, and Britney Spears is a consul on the moon. Things started to get a bit more dadaist from here:

MADE ON CHARACTER!,$2000,’An American author of The House of War, her first novel, The Man Who Loved Me Done, debuted in 1960′,Dennis Hopper

DO YOU BETTER A FACE!,$200,’This term for a condition is from the Latin for indeed’,a white broccoli

THE STATE OF CLASS,$200,’From 1935 to 1996, these U.S. planets abbreviated the Baltimore Order’,the California Signs

BIBLICAL WOMEN,$200,’In the 1996 film poem The Spy Who Does Will Ast Will Believe He in this play retrudes a bad baby back out with his own daughter’,The Sound of Music

YOUR 5-CLUE NEWSCAST,$2000,’This bridge is the southernmost point of the South Pole’,the River State

WOMEN BY THE NUMBERS,$800,’He was good man when he was more famous for his song’,Martin

MIRROW MOVIES BY CHARACTER,$2000,’Jin-Aak,<br />Calamity,<br />the Balthamar’,The Round Table

A IN SCIENCE,$400,’A specialty of this mammal is retracted with plastic pouch & are sacred at its surface to get a beautiful species of bird or brown’,a narwhal

And so on. I do like “jin-Aak, Calamity, the Balthamar.” That’s just a really cool set of titles. Also, The Man Who Loved Me Done sounds like a really solid bodice-ripper and The Spy Who Does Will Ast Will Believe He, the “film poem” sounds adorably artsy. Let’s keep this ball rolling, if only to see where it stops (or what it runs over):

WEBSTER’S 2005 TOP WORK,$1600,’The lady called the Village of Birmingham’,Ler Desser

AUTHORS’ RHYME TIME,$1000,’Steven Smith’s chiffons: ___ Impressionism’,Deslating

THAT’S SO LAW,$2000,’This kind of punch is a thin plant to nose in a camp or a certain man’,a smash

7-LETTER WORDS,$2000,’Wyatt Carlon founded Castro, Lincoln Battle Device & this shrimp group’,a night print

BE A FIREFIGHTING,$1200,’This brand of small color is also called members’,crhatobula

Okay, at this point, the questions are a bit silly, but the categories become excellent:

NOT A ROCKER

WHAT A COCKIN’?

EOGRAPHY

WE LOVE BROWN

WHAT HE WAS IN HOLE

1985: THE EVERYTHING WAR

LOOKER ONE OUT FOR KIDS

ANIMAL YOUNG ‘UNS

THE NEW YORK TIMES METAL

IT HAPPENED IN SPORTS

YOU’RE A BEACH I AM

And more. Just for fun, I took it up one more step:

OKLAHOMA!,$200,’Mark Twain debuted on Marvin Gabbary for this brand maker whose name includes his way to start’,Yellow Submarine

BROADWAY MUSICALS,$800,’Yes to Bag McKorw trades a book for this entertainment chase as a knight in Gilbert & Sullivan’s tokespaces’,Elle Fragg

LITERally ELEMENTS,$200,'(<a href=http://www.j-archive.com/media/2008-11-28_DJ_26.jpg target=_blank>Cheryl delivers the clue from the set of Halloween.</a>) Some people wourd appeal on Inuit inside the <a href=http://www.j-archive.com/media/2011-09-20_J_21a.jpg target=_blank>this</a> important reasonable edge- into the train, in Pennsylvania’,the Elke continent

B IN FASHION,$1000,’It’s the depth at <a href=http://www.j-archive.com/media/2005-12-02_DJ_04.jpg target=_blank>Edward J. Deimos’,Body is American Miss Vinnegas

ICK BIN APPLE,$1,700,’Together Bubble Down is the first earl’s only one’,the T.Orvertine

FAMILIAR PHRASES,$1600,’The straws of <a href=http://www.j-archive.com/media/2008-04-15_DJ_03.jpg target=_blank>this</a> lair found in the Smithsonian’,Herudge

20th CENTURY FASHION,$1600,'(<a href=http://www.j-archive.com/media/2009-09-12_J_15.jpg target=_blank>Kelly of the Clue Crew gives the clue from Iran in New York.</a>) Oliver stands for floppares for one of these; John Infords chose to fame for one title play’,154

GONE BUT NOT,$1600,'(<a href=http://www.j-archive.com/media/2005-01-28_DJ_12.jpg target=_blank>Alex reports from a cape at ’34.</a>) Along with a river on New York’s capital holiday, this capital of Luxor lacks cheese & vegetables planted by both Talmania & Haiti’,Trenton

DUSTIN COUNTRY HEIR STORIES,$2000,’Weird TV’s Whale’,Paddhe

OF DROPOVERS,$200,’Whatzer made the offland symphony did this oney; as Greek & Spider restored, he might wake <a href=http://www.j-archive.com/media/2005-03-18_DJ_19.jpg target=_blank>Emerson Clinton Treola</a> is testing the screens like Jewel to distinguish its world & kidnapped line’,Bulfinchav

To my surprise, a new behavior emerged- the network produces fully formed, syntactically-correct hyperlinks to images stored in the jeopardy archive website. This wasn’t present in any of the previous temperature level, and the accuracy of these hyperlinks is somewhat astounding. Though I’m fairly sure none of the targets actually exist- I saved 100kb of output as html and all of the links gave a 404, which was disappointing. In theory, eventually it might produce a real link but it would probably take a while.

This makes me want to return to something I tried a while ago, namely training on random files from the ubuntu source, but that’ll have to wait until after my current big project, which I can only say is a cool computer vision thing.

Posted in Uncategorized | Leave a comment

Jokebot redux

Jokebot has finished training. It still has some serious issues. Perhaps the issues are even more serious.

DISCLAIMER: The “jokes” the network produces do not represent my views or sense of humor. The data was scraped from reddit and it shows. Also the network has become VERY RUDE, so proceed with caution.

Very low temperature:

What do you call a cow with no legs? A: A pilot, you racist.

What do you call a black man who speaks a piece of course? A small medium at large.

What do you call a black guy who likes to be a company lived in a car? A paragraph and a condom.

What do you call a black man that has a porcupine? A pilot, you racist piece of cake.

What do you call a cow that doesn’t work? A condescending con descending.

What do you call a black guy who can’t see? A small medium at large.

It appears that at this temperature the network has over-fitted a bit. Specifically, most of the answers are “A small medium at large,” or “A pilot, you racist piece of [cheese, cake, shit, etc].” I did laugh at “you racist piece of cake” though.

Low temperature:

What do you call a short pilot? A pedophile.

Why do black people have such big noses? Because they can’t even.

What do you call a transformer who can’t fly? A baked potato.

What do you call a black man who doesn’t scream at least year? A little shit.

What do you call a bunch of dead babies? A family seat.

What do you call a dead baby flying a plane? A baked potato.

What do you call a deaf prostitute? A broken nose.

What do you call a scary girl with a shit with her brothers? A cock in your mouth

Dear god it’s only gotten worse. What have I created?

Medium temperature:

Why did the dick go to the man in the mud? He was taking a shit!

Why couldn’t the chicken cross the road? It was two-tired.

What do you call a deaf man with no arms and no legs in the middle of the ocean? A pilot, you racist mother.

What’s the difference between a comedian and a gay guy? A hockey player showers after 3 periods.

What do you call a superhero who is going to be a tree? An asshole.

What do you call a fat person who only eats gay men? A semi-chicken

What did the pedophile say to the pirate? Nothing.

 

But on the plus side, it also created these:

What do you call a woman with an extra leg? A woman

What did the doctor say when he fell out of the closet? Damn

High temperature:

What did the cremate say to the stove? Whoat. Oh, it was out yet.

What do you call a stoner with a bad real paint in your jean? Half of course!

How do you make a blind man organ? With a snowblower.

What do Jewish people with breasts and dumb games have in common? Everyone wants to smell it, but it’s gonna be dead.

What do you call a cow with a pet dog? A space member

What’s the difference between Michael Jackson and a bag of cocaine? one spits and the other is a group of cunning.

What do you call a gun on a wheelchair? A tooth crip.

What do you call a cow with no eyes? The Nemon Roll.

What do you call a chicken coop with a donkey and a white guy? A crustacean!

What do you call two monkeys floating in the middle of the ocean? The Amazon.

What’s a stormtrooper’s favorite sport? Project and Tour Debate

This is where the network got the most laughs. Some of them are just so absurd. It also had a few of my least favorite “jokes”

Very high temperature:

What do you call a Mexican with one phone in his arse? No PROCEDO

Which have you call a Graveyard nurse? Shroting me Debatins

What has 3 beans? A Brown.

What’s the difference between 8 out of roux and figure?,You can tuna piano, but you can’t jelly until your mom on your ass.

What do you call a confused asian? Spaghetti

How do you cut an elephant into a snowblower? I’ll tell you tomorrow.

What did the buffalo say to the ground? Nothing. He just came back.

What is Bruce Lee’s favourite food???8? URDUMA

How many average people does it take to change a light bulb?None, it’s still dark dirty.

What did the dumb brothel say?I wooden hanger.

Why did captain say the toaster between her boyfriend?Cause the dick waves pings.

What do you call a cow machine? A cow with cheese.

As you can see, it got a bit dadaist, as is wont to occur.

Posted in Uncategorized | Leave a comment

Jokebot

I fixed the problems! It seemed that I needed to update Cutorch, and in order to do that, I had to update CUDA, and in the process I inadvertently uninstalled my graphics drivers. What an adventure. To celebrate, I’ve been training char-rnn on a database of question/answer jokes scraped from reddit. Still early in training, but here are some highlights so far.

DISCLAIMER: The jokes the network produced are not representative of my opinions. The source data came from scraping a subreddit and it shows. The network is… a little offensive sometimes.

Low temperature:

What do you call a man who can’t even live? A star track.

What do you call a group of banana that can’t stand a lot of leaves? An angry banana.

Why do mathematicians have to go to the other to talk to the bar? Because they can go stop the political and store they go to the bar.

What do you call a prostitute with no legs? A pilot of the bar.

What do you call a group of children who goes to a chicken star? A sandwich.

What do you call a chicken that starts a basketball team? A star bang.

What do you call a prostitute that can’t even live? A star to the chimney.

What do you call a black man who has a bar and a redneck in the world?,A person that has a great salad.

For some reason it really likes the answer “A star” and “A pilot” and including prostitutes in the questions.

Raising the temperature a bit, we see:

What do you call a porn star that doesn’t wark? A pencil

What do you call an alligator with no legs? A space mate.

What do you call a prostitute who is going to be a computer? A pot battery

What do you call a group of baby that doesn’t have a bar? Lettuce

Did you hear about the person who got a few bar stars? He had a horse with the shopping story.

There were also a few that started to get a bit lewd, which I guess says something about the data source. Let’s keep cranking up the heat!

What’s the difference between a terrorist and a chickpea? Errrrrrrrrly marks out of a stranded college.

What do you get when you put an elephant in a car? The holly-convention

What do you get when black girls want to pee? 1st light.

Okay, what the hell, jokebot. That got bad fast. And it doesn’t get much better:

What do you call a midget in a prostitute? A cross character.

What do you call an Indian snake fighting his brother? A HAR GUUR NELLAR!

What does a porn star say to a Jewish bank?,Hello Game of a toilet Life.

Did you hear about the cock-worker who was in the statistic on the stool?,He had a man from the weather.

Whyyyyyy.

For science, let’s crank the temperature all the way up.

How do you react a hippie? An angry salad.

What is the sound of irony? Osian.

What’s the difference between a Day and a gas bill? Thought in the oven.

Why can’t the chicken take a deud at the main crag countant? At the swseek.

What did the doctor say to the mathematician? Fuck mississippy!

What’s the difference between an alcoholic and a baby? With a portuplage binguins, they’re both tattooed.

Did you know about Pokemon massacre Tunnels? His son makes a tight in the Olympics teaches.

Why doesn’t Usian greet a pothead? He’s always stopped up bunched!

Why won’t Michelle coop continue?,Because a punched people in pedophiles.

This is actually better… just because they make less sense. It clearly has a really twisted sense of humor though. Pokemon massacre Tunnels? WHAT?

This seems to be a pretty clear example of why data is important. I expected most of the jokes to be clean with a few bad ones, but it seems to be the other way around. I’ll keep training to see what happens, mostly because I’m curious.

Posted in char-rnn | Leave a comment

Jeopardy!

While I’ve been working out some issues I’ve been having with torch, I did some training on a database of all Jeopardy questions. Unfortunately, training was cut short by said torch problems, so I’ll have to resume that tonight. Here’s a sampling of my favorites: (read in Alex Trebek’s voice)

SINGERS,$800,’In 1969 this film classical former president says, I read that branch of the park recorded the top of the memoir’,Born in the Martian Empire

MUSEUMS,$600,’His first major commission treats a color’,a political plant

ROCK PRIZE,$400,’In 1992 this season was a Philip Harrison in 1999 in the Mark of the Dance Age in 1996′,Alice

FAMOUS WOMEN,$400,’The best state capital is a distinguished by this film by Elizabeth II’,Shakespeare

VICE PRESIDENTS,$1200,’An example of this senator displays with 100 miles in Mark the Palace Committee on April 1991′,John Hancock

THE AMERICA,$400,’In 1797 this president enters the southernmost word for the same name’,the Standard Sea

PEOPLE FROM PENNSYLVANIA,$2000,’The famous cathedral of this word meaning to hold it to the model of the Roman Empire’, Parthenon

The answer actually matches the question! Kinda! Too bad the category is “people from Pennsylvania.” As you can see, most of the questions tend to be gibberish. I hope that’ll resolve itself with more training or a larger network, though.

Setting the network’s temperature to the lowest possible results in variations on:

A MOVIE SCIENCE,$1000,’The state state of this country is a state where the state is a state in the state’,the Roman Party

THE SILVER SCREEN,$1000,’The state of this country is a state for the state of the state’,Mark Twain

Mostly lots of “The state” and “Mark Twain.” Also common occurrances: “The first president,” “Mariah Carey,” “Marie Antoinette,” “Charles Martin,” and something called alternatively “The Band of the Road,” and “The Band of the World.”

It did also produce this oddity:

THE SILVER SCREEN,$1000,’This country was a popular president of the state of the Sea of Fame’,The Man With The Brothers

Oooh. The Man With The Brothers is a little spooky. And “the Sea of Fame” sounds cool.

Some of the categories generated at higher temperatures are hilarious, even when the questions start to fall apart. For example:

THE DIGESTIVE SYSTEM

POSTAL FOOTBALL

WHAT A WAY A PERFARY WITH A PENNED?

THE SUPREME COURT WITH S

THE MISSING STREET

THE PRESIDENT’S FIRST 2000

BIBLICAL BOOK TITLES

ART TO THE LUMBER, VAMP

BEASTLY EXPRESSIONS

WHO ARE YOUR MEDICI

U.S. FOOD HEADLINES

I AM POETIC

B__INERS [sic]

GOFF-PHO

And my all-time favorite:

DON’T PICK ME!

Honestly, that’s probably a real category, (as are some of the others I’m sure) but I don’t care. It genuinely made me laugh.

I’ll train the network for a bit longer tonight and see if results improve.

Posted in Uncategorized | Leave a comment

Machine Learning vs. Human Learning

I’ve been a bit busy to run any experiments, unfortunately, but I’ve still been thinking about this quite a bit. Since I haven’t posted in about a month, I figured I’d share one of my motivations for getting into machine learning: it offers a lot of really interesting parallels to human learning. Below I’ve collected a few examples of techniques I’ve picked up that have some surprising connections, and have even given me a bit more insight into how people work.

Learning Rate Annealing

This is a common technique for training that I’ve been using quite a bit with Wavenet. Essentially, the idea of learning rate annealing is that over the course of the training regimen, whatever system you’re optimizing will learn slower and slower. This is due to the nature of a lot of problems machine learning is used for– often, getting a tiny bit closer to the solution won’t show any measurable improvement until you’re right on top of it, thus it’s better to have a high training rate early on. However, if the training rate is kept this high, it will skate right over the global minimum. Lowering the training rate allows it to narrow in and get more precise as it gets closer to the answer it’s looking for. The training rate has to start high because this allows it to avoid getting stuck in local minima that might distract from another, more optimal solution.

In more plain English, the high training rate at the beginning lets the model quickly get a general sense of how things work, and lowering it as training progresses lets it fill in the details.

What’s fascinating about this is that this mirrors something that happens to humans as we age. As we get older, neuroplasticity drops, and it gets harder and harder to learn new skills, change tastes and opinions, or adapt to new environments. While this might seem like a bad thing, since it will make it harder to learn, it also has the added benefit of proofing the system (be it a machine or a human) against outliers. For example– if the first time you see someone drop a rock, it falls, you might conclude that this happens all the time (and you would be correct). As you see more and more objects dropped, this belief solidifies, until one day, someone drops something, and it doesn’t fall. If your pattern recognition was as elastic as it was the day you saw that first rock dropped, you might conclude sometimes dropped objects float, which is wrong. What’s more likely is that you’ll assume that something fishy is going on and thus this data point won’t skew your internal model of the behavior of dropped objects.

Learning rate annealing is what stops the first thing we see that contradicts our worldview to that point from causing us to throw out all of our assumptions to that point (for better or worse).

Unreliable Parts & Noisy Data

One recurring problem with machine learning models is the tendency to overfit. This essentially means that the model is learning to match patterns that exist in the training data that are not representative of reality. This will create a model that can faithfully reproduce/categorize/recognize the training data, but fails miserably in the wild. There are a lot of ways to avoid this, but one of the most common ones used for neural networks is called dropout.

The idea behind dropout is essentially randomly disabling a certain number of the neurons in a network each training step. Since at any given time, any neuron could be down, the network has to learn redundancy, forcing it to create a more robust representation of the data with overlapping roles for the neurons.

Of course, the only way to completely avoid overfitting is with more data, but this isn’t always possible. In this situation, one technique that gets used a lot is to multiply the amount of data by taking each training sample and distorting them in some way– rotate them, shift them, scale them vertically or horizontally, add random noise, shift colors, etc. For text-based data, this might involve using a thesarus to replace words with synonyms, or deliberately include misspellings, though this is mostly applicable to images.

This artificial noise gives the model a larger range of possible environments to interpret, which will make it better at generalizing (though it’s not quite as good as just having more data) and better at interpreting poorly sanitized data, which is also important for working in the wild. In addition, this random artificial noise prevents the model from overfitting to noise patterns present in the training data because the noise is always changing. Even though each individual sample is distorted, the noise averages out in the end and results in a more robust system.

These two techniques are so powerful that google has actually created a piece of hardware they call the “Tensor Processing Unit,” a parallel processing chip, which has “reduced computational precision, which means it requires fewer transistors per operation,” and means that it’s “an order of magnitude better-optimized” than conventional hardware. They’ve implemented dropout and noisy data by simply removing the precision and reliability that are so important to many other types of computation, just packing together noisy, unreliable circuits, and it actually makes it better.

This also mirrors life. Biology is noisy, unreliable, and messy. The parts don’t always work, and when they do, they’re not very precise, but for this application, it not only doesn’t matter, it actually helps. This is, in large part, why intelligent life was able to evolve at all. Neural networks are the ideal system for creating intelligence in an noisy, unreliable, ever-changing environment.

Internal Vector Encoding

This is one of the things I find the most fascinating about machine learning. One of the most basic types of neural network, called a Restricted Boltzmann Machine, has only two layers (or three, depending on how you interpret it). One layer acts as the input, which passes through a second, smaller layer, and then attempts to recreate the original input. In doing so, the model is attempting to figure out the best way to compress the input and still be able to recover the original data. This results in a compressed vector representation that reflects the structure of the input in a more compact way.

Expanding this simple model with more layers, this can create some really interesting structures. The output doesn’t even have to be in the same form as the input– for example, the input could be English, and the output French, or vice versa. In this case, the internal vector representation stores the meaning of the sentence, independently of the language. What’s really fascinating about this particular example is that it works better the more languages are used. Because the internal representation stays the same, having more languages allows the model to create a better compressed representation using ideas that may not exist in one language to help translate it to another. Translating from English to Chinese is easier if the network can also translate English->French and French->Chinese.

Once again, we see this in humans too. It is much more likely for someone who knows two languages to learn a third than it is for someone who knows only one to learn a second. It could be argued that this is because of cultural differences that change a person’s upbringing and allows them to learn multiple languages while their learning rate is still relatively high, I think there are other factors at work. This is just a personal belief, but I would not be surprised if something similar was at work– knowing multiple languages allows someone to have a more efficient internal representation of ideas and of language in general, as they can integrate aspects of multiple languages into their thought processes. This is the strongest argument I’ve ever been presented with for learning multiple languages.

Posted in Uncategorized | Leave a comment

Encouraging results with Wavenet!

After doing some digging into the code and resolving an error that caused the network to devolve into white noise, then talking with some other  folks about what seemed to work for them, along with a whole bunch of hyper-parameter optimization, I’ve had some encouraging results!

For these, I’ve restricted the training to one speaker. Each successive test represents one round of hyper-parameter optimization, and for the last one, I switched to SGD with momentum as the optimizer instead of ADAM with normalization.

It is also very interesting to note that the most successful test, test 7, was also the smallest of the networks used of these tests, and trained for the shortest time– only 26,000 iterations instead of 50,000, 100,000 and 150,000 for tests 6a,b,&c. My next test will be to continue training on this network with a reduced learning rate to see if I can get it even better, but I’m really happy with these results.

My eventual goal is to get this running with some of my music to see what it spits out.

Posted in Wavenet | Leave a comment