MtG Card Database Maker

13 posts / 0 new
Last post
I am not sure if anyone will find this useful at all but I am about to finish up a very rough program that will create a database of all the cards from magic and store all the information into an access database. It pulls all the card information from the gatherer and stores multiverseid, card name, converted mana cost, type, power, toughness, expansion, rarity, and the url to the card image. It is also capable of pulling the legality of each card and putting it into the database as well (will add A LOT of time to the runtime). I am also hopeing to pull the card text as well (this is next on my todo list.) My thinking was that you can use this database for any application that you want to make and you will have all the cards you want, and you can just run it again when the next expansion comes out.

Please let me know if this is helpful at all.

Source code for application is here: github.com/w9jds/MtG-GathererDBMaker

Thanks.
From the looks of that, you're only getting multiverse IDs up to 1500, leaving the database woefully incomplete. In fact, if you're just iterating over multiverse ID, you currently have to go up beyond 360,000, and fetch a page for every single one of them (actually you don't need to fetch the numbers that aren't assigned, but good luck finding out which ones those are any other way). Not only that, but doing so will create multiple redundant entries for reprints of the same card, and even create entries for foreign cards which you're even less likely to need.

My approach is to hit up a single page: gatherer.wizards.com/Pages/Search/Defaul...[m//] From that, you can easily parse all the same information, including card text, with a lot less bandwidth. It doesn't give formats in that layout, but you might also consider calculating format legality yourself, based on the set history you can read there as modified by a pre-programmed set of ban lists.
Yea I was planning on actually asking the user what multiverseid numbers they wanted to run, I just had the 1 - 1500 there for testing purposes. As for the duplicates I wanted the user to be able to select the different prints of the cards to make the list of cards on here more accurate. I however didn't know about the foren ones. I will have to look into that! thank you. 

Nifty idea, but I would suggest using SQL, not Access.
Access "works" for certain things, but SQL is way more powerful and less of a pain to use.
SQL has a slightly harder learning curve, but once you "get" it you don't have to fight the software so much.

I did something like this years ago, and had to re-write the query every time, because Access would "optimize" it after I finished, and scramble the whole thing beyond human recognition.


SQL Express is free to download and use from Microsoft, so that's not an issue either.
Interfaces with .NET pretty easily too, so no worries there either.




Also, I'm not trying to steal your thunder, but I've been maintaining an offline card program for 5 years now, and I share the database with anyone who wants it.  

If you're just doing this as a learning experience, go for it. I'm just pointing out that if you don't want to spend the time to do screen scraping yourself, there are other solutions.

Mine doesn't pay any attention to Multiverse ID though. 

Link in my sig, you could just download and rip the DB out of the source files (/Resources/Data)  or download the Excel backup file.  It's slightly out of date now, but I'll update it this weekend with Gatecrash.

Happy Coding!  :D 

Casual Magic player since 2003 (Onslaught Block). 60% Johnny, 40% Timmy. Want a free, graphics-based, collection database to inventory your cards? I made one! Feedback welcome. Program runs offline, includes powerful search options, art, Oracle text, data import/export, and a rigorously updated list of every card ever printed. Version 5.13 (Theros) now available!

So. Green needs an iconic creature type, eh? How about wurms!

One Billion Words - 1001 Fantasy Landscapes: Share DnD-ish landscapes for use in homebrew campaigns!

Yea I saw your application! I was doing this mostly for the experiance and because I thought it would be fun. I haven't looked into SQL express that much so I am going to go give it try right now.

Thanks! :D 
I also wrote myself my own application. I purchsed MTG Studio at first and was using it. But since I'm a software engineer I found myself wanting to do things not available so I took the task of creating something on my own.

I use SQL Server 2012 as my back-end database and Visual Studio 2012 C# as my development tool.

I manually enter new information about the card set itself
Set Name, Release date, Set Abbreviation.
I then have an import screen where you select the card set's you want to import.
It then imports the basic card set information from Wizards Gatherer site.
Multiverseid, Card Number, Name, Artist, Rarity, Color is for display purpose only on my import screen
It then spawns off threads for each card and get's detailed information for each card using the multiverseid.
I also obtain the different languages
 
Additional calls are then made to magiccards.info website using the multiverseid for each language obtaining the card text, image for each language.

I have tables to keep track of all cards, and then the cards I own.
I can filter on a variety of items


 

I also can build decks, enter foreign cards, update prices, sort my results on any column.
Even group the column information, on 1 or more columns

I can add cards to my inventory. I just enter the qty for each card. Once I'm done entering all quantities I just select add to collection.

I change change the language to a different language and then any card added to my collection is added in this language. When viewed and selected it will show the language of the card in my inventory. 
I can also click a button next to the card if it's a foreign language and it will display the english equivalent.
It show's Card Abilities (Also can filter on abilities)
Legality (Can also filter on Legality)
Next to the image are 4 buttons
Update Price
Jumps to ebay using this card as a search criteria
Jumps to Magiccards.info website for this card
Jumps to TCGPlayer.com 

I can create a Buy list of cards I want
Build a Deck using cards from my collection as well as all available magic cards and then create a list of cards needed to build this deck. 
So if I create a deck with 2 Leyline of Sanctity and only own 1 of these cards it will show I need one of these cards.

This application keeps evolving. I keep adding new features.
Just recently I added Multicolor Filter.
This filter let's you choose a guild, or custom (great for EDH)
It will return cards for this guild or is I select custom and pick Black, Blue, White it will return
Black, Blue and White cards
Multicolor Black/Blue, Black/White, and White/Blue cards
Gold Border - Black and Blue, Black and White, White and Blue, and Black, White and Blue mana cost cards
Add Legality of Standard and only Standard Legal cards are displayed.
Including cards from earlier editions that are re-printed in the Standard set's
 
I'll gladly tell you more about everything I've done.

Joe 
That is the same reason I wanted to make one, I'm trying to get a job as a software engineer and thought it would be fun and good experiance to create my own application. I really appreciate all of you post here with information about your applications
That is the same reason I wanted to make one, I'm trying to get a job as a software engineer and thought it would be fun and good experiance to create my own application. I really appreciate all of you post here with information about your applications



Take notes on the things you learn.  Seriously, after a while you forget as they become second nature.

I know I learned a lot about high-level project management and long-term planning, mostly by trial-and-error.  These are the things that are hard to teach at school, since the confined and pre-determined nature of a school caricculum make it hard to get imersed enough to truly experience.


For example, I learned quite a few tricks about writing open-ended code, so you can leave room for future additions.  My first version of the program was all hardcoded into a single module, but as I started adding features I extracted each chunk of code into its own class.

I talked about that quite a bit during some of my job interviews, and discussed the thought process behind how the code base was organized.  DB queries.  GUI.  Search controlers.  File read/write management.  Each of these is a clearly divisible function.

In these interviews, I freely admitted that my first version was fairly bad, but I kept improving it as I learned more in school, and from reading books and web pages.  Each job will have it's own programming environment, but showing you are willing and able to learn and adapt is a huge advantage.  

Many people can recite technical details, but are hard-pressed to implement them, or adapt a good idea for a different implementation.



Another thing I would suggest is to get some hard facts.

"I wrote a program to that does stuff for a game I play" sounds very childish.  People unfamiliar with Magic and it's complexity won't appreciate the effort you put into it.

"I wrote an inventory management application for a collectible card game I've played for years" sounds better.  Re-inforce that with speciffics.  20,000 unique cards; 100,000 unique printings; a robust search engine with 50 search paramaters.  Dynamic queries to check tournament legality as cards rotate in and out of various tournament formats.

All of those details imply a lot of complexity behind the scenes, and even people unfamiliar with the game should be able to appreciate the scope of such a project.  



Good luck!  ^_^

Casual Magic player since 2003 (Onslaught Block). 60% Johnny, 40% Timmy. Want a free, graphics-based, collection database to inventory your cards? I made one! Feedback welcome. Program runs offline, includes powerful search options, art, Oracle text, data import/export, and a rigorously updated list of every card ever printed. Version 5.13 (Theros) now available!

So. Green needs an iconic creature type, eh? How about wurms!

One Billion Words - 1001 Fantasy Landscapes: Share DnD-ish landscapes for use in homebrew campaigns!

I really appreciate that advice! I'll make sure to update here on my application as I get farther into it. 
Also for anyone interested I started the work on the actual application and have started pushing it to github when I am able.

It is located here: github.com/w9jds/MagicManager
I have updated the database maker! It now filters out all of the foren cards so it only adds cards that are in english. Also it allows you to select the multiverse ids that you want to add. I also went and made it so it skips the non existant ids faster
Does it allow output in .mwBase format (for The Forbidden Program?)
Does it allow output in .mwBase format (for The Forbidden Program?)

At the moment I have only tested it on .mdb formats so far. Sorry
Sign In to post comments