What better way to further our adventures then with some good old fashioned gold farming, and with the advent of blockchain gaming there is no better time then now to get involved with some bot development.
A game bot requires three things; vision, control output and a decision mechanism. Below you will find all the required tools you need to develop a state of the art Machine learning computer vision ROBOT, to farm MMO gold!!
Lets start at the start and that is tensorflow the robots “brain”, learn all you can here before you move on then you are ready to develop your bot. A bot needs a good brain ya?
A must have, you will not get far without this, your arms and legs Mr Bot. Lets you move the mouse, type and click (Don’t forget to insure some randomization in your implementation to avoid that pesky anti-bot detection you may face).
So you have your “brain” and your “appendages” the only thing missing is your “eyes” well openCV (Open Computer Vision) is the worlds finest eyes for your bot, just like tensorflow if you truly want to be a bot master you must learn all you can about openCV.
So you have everything ya? Your brain and legs, arms and eyes!! But let me tell you while you are making a bot you will want to read the screen for text to get variables and such, this is the best way to do it with tesseract like the above if you learn this well you will be equipped to be the master of bots.
Easily train your image recognition models online then export to any format you need, no point having the “brain” if you have no pre established concepts.
The rest appear in no particular order, these are useful but not essentiual:
Decent wrapper library for Google vision OCR to find position of a word/line/para in a image.
Here we have a Nodejs native extension to search “subimage” inside an image using openCV get the position and then you can easily click on it, bobs your uncle, you have a bot. It’s just a simple algorithm to find the most prominent color in pictures but extremely useful.
Running out of options? A node.js app to find shapes and blobs in an image.
A bot needs good eyes and this I could not have named this better my self YOLO, this addon allows you to use a state-of-the-art, real-time object detection system called Yolo. This addon came out from a computer engineering final project, VAPi (You only look once).
If you are lazy and want to use a API solution here we have Google Vision, its great to jump right in and get started but keep in mind it is a hosted solution, this one is another option for your “eyes”.
A little native OCR implementation that uses Tesseract OCR (Optical character recognition). In other words this turns images into text strings.
Another nice OCR library this one uses Tesseract and OpenCV.
Some video’s done in python but still great for learning, maybe one day i will make some for NodeJS but until that day comes enjoy these:
A Gift for reading this far:
Now these 2 are very very cool and basically allow you to teach a AI to play any game with no work at all. I thought maybe they are to powerful for you plebs but in the spirit of freedom of information here: (these 2 are python and not nodejs):
Start it and play your game…. amazing right !
Serpent.AI is a simple yet powerful, novel framework to assist developers in the creation of game agents. Turn ANY video game you own into a sandbox environment ripe for experimentation, all with familiar Python code. The framework’s raison d’être is first and foremost to provide a valuable tool for Machine Learning & AI research. It also turns out to be ridiculously fun to use as a hobbyist! (yep you read that correctly).