By Lisa Shock on Thursday, November 23, 2000 - 08:51 pm:
I have always been interested in the shrine and the process it goes through to make monsters (or cards in BC) and their stats. I am also intrigued that virtually every blow-hard that I meet in real-life who claims to have ‘beat' the game in three days or less, also claims to know what the games read, and to be able to make monsters at will with their CD burner. After one particularly distasteful encounter with teen who blustered that there was no way I could ever learn as much as he knew about the game (but he couldn't make correct change at the cash register), I decided to look into what exactly the game ‘reads' from CDs.
I started by thinking about the basics, the game is stuck with the PSX system to use for the process; it has limited RAM (2 megs), double speed CD drive, a fairly limited amount of time (although PSX and music CDs are read faster than CD-ROMs and GD-ROMs ), and a pretty old processor. The game uses a limited number of monsters, created from frames and skins pre-set by the programmers.
Then I got a major break; Battle Cards Episode II was released, and several of us quickly noticed that the shrine viewed certain CDs as identical to one another. So, I began to look at these ‘identical' CDs to see what they had in common.
Here are the essential facts:
a) If the lab in BC viewed the CDs as identical, they made identical monsters to each other in MR1, and MR2, and even the stats would be the same.
b) Non-identical CDs that made the same monster in MR1, did not always make the same monster as each other in MR2 - and, the stats would vary.
c) CDs could be ‘identical' yet very different sorts of media, for example "Bust A Move 4" for PSX is ‘identical' to "Backdraft" a music CD soundtrack.
d) If a CD has only one track, it is a pure-breed in MR1.
So, I began looking for specific characteristics that a PSX game could share with a music CD. The first clues seem to indicate that the number of tracks, total minutes, and total seconds was the answer. "BAM4" and "Backdraft" had all three of these numbers identical to each other. And so did all of the other ‘identical' pairs. I looked at more of my CDs and found that many CDs that made one breed or another, had one of the three stats the same. I started working on charts to show which numbers made which monsters.
Then Torey gave me some numbers for particular CDs that made particular monsters, which completely contradicted my charts. Then I made a critical discovery; the numbers that the PSX Cd player displays as number of tracks, total minutes, total seconds can be very different from the number from another CD player, say, in a stereo or in my computer's CD player! I began using the PSX CD player exclusively for data, and thought I was making headway...until I hit a huge obstacle! I had two copies of Christmas with the Chipmunks, one a Columbia House pressing, the other a BMG music club pressing, both had the exact numbers in the CD player, yet they made completely different monsters! One was the rare Satan Clause (Wracky/???) The other makes a pure Gaboo.
I then decided to look into what exactly CD players can read when they look at a disc. It turns out there are several types of hidden files encoded on CDs that players can read. Some of the files are just redundant data to help prevent skipping. Some tell the player where exactly the tracks begin and end. Others encode data packets that allow players to individually identify particular CDs -these are useful in players that allow users to input titles and such, for special displays on more expensive players. For more details, click here.
I searched for software that would display subcode data on my computer screen, and found a program called CloneCD, which has been useful - if only in showing me the real complexity of the problem.
I looked at few CDs, and found that ‘identical' CDs have truly identical subcodes. As for others, I have compared CDs that make the same monsters and have not been able to crack it. My best guess is that the data packet in the Q sector of the subcode, usually used for those high-end players, gives the game info which the game then checks against a chart. I have not been able to make much headway in the chart beyond my original tracks/minutes/seconds correlations.
Another minor point: I did find out why DC GD-ROMs mostly make Gaboos, and so many of them are identical in MR. All the DC GD-ROMs are read by the PSX CD player as having 2 tracks, with usually less than 2 minutes total time on them. (Unless they have additional graphics files on them, like Sonic Adventure.) Co-incidentally, all DC GDs have a brief audio track warning users to remove them from music players. Each manufacturer uses a unique voice actor to make this message, but the clip is used over and over again. Thus, all Capcom games have identical, brief, audio tracks - increasing the likelyhood of identicality.
Here's a sample: the entire subcode data from a one-track Dreamcast GD-ROM. (Resident Evil: Code Veronica, both 'identical' discs 1 & 2)
By torey_luvullo on Friday, November 24, 2000 - 02:37 pm:
as you know, ive been following this study in the background. i know you have made massive charts attempting to correlate what tracks/minutes/seconds correspond to which monsters in which games. i know that a perfect correlation was not achieved - how close is it tho? 75% accurate?
the practical application would be to allow those looking for certain monsters to snap up bargain basement cds based on known info. even if there is only an increased chance, hell, we face down oakleyman; we can deal with frustration.
...or isnt it that simple?
By Lisa Shock on Friday, November 24, 2000 - 06:50 pm:
They seem to fall into groups in about 50% the zone, in other words, about half of my CDs making one particular type have identical Track/Min./Sec. The difficult part is that the info on the outside label is not in sync with what the PSX says is the actual data. Sometimes it's a second or two, sometimes minutes of difference. So it would be hard to judge based on label info.
I mostly looked at common monsters for charts, because I had more CDs to compare. My one pure Undine didn't seem like a large enough sample. I could post the most common data, for reference - I just don't want people upset with me when a CD doesn't make what the chart says.
By torey_luvullo on Saturday, November 25, 2000 - 11:33 am:
well, even with cd lists there are instances where different pressings of the same cd gives different monsters. maybe a walkthrough of your largest monster type chart, with appropriate cautions and disclaimers, would be the easiest way to grasp what you've accomplished - if that wouldnt be too hard for you to do, that is...
By Lisa Shock on Saturday, November 25, 2000 - 11:14 pm:
Some pressing differences are easy to spot; Beck Mellow Gold and Mariah Carey Christmas each have versions with differing numbers of tracks. (Although there are several versions of The MC album with the correct # of tracks, and only one makes the rare. I don't have the rare yet, only 2 different wrong pressings!) Some wrong pressings are un-notice-ably different, until the subcode is viewed, like my 2 Chipmunks CDs. I'll see if I have time to type in a chart or two.
By torey_luvullo on Sunday, November 26, 2000 - 06:44 pm:
take your time, lisa - this question has been intriguing me for a couple of years now - it doesnt have to be solved overnight!
By Lisa Shock on Monday, November 27, 2000 - 02:58 pm:
Well, here's one tantalizing bit of info:
CD title
Tracks
Minutes
Seconds
MR1
MR2
BC
Mr. Driller PSX
1
35
06
Dino/Dino
Jill/Jill
Mocchi Hide
Tiger Woods PSX
1
35
16
Dino/Dino
Jill/Jill
Worm Emerge
The Smiths Interview CD
1
35
19
Dino/Dino
Jill/Jill
Gali Scratch
Williams Arcade #1 PSX
2
35
48
Golem/Jell
Jill/Pixie
Durahan Thunderbolt
Velvet Underground VU
9
36
38
Golem/Pixie
Jill/Pixie
Monol Flattening
Galaga PSX
8
38
56
Jell/Worm
Jill/Hare
Mocchi Roll
The pure Jills have Tracks and Minutes & MR1 and MR2 monsters the same, the Jill/Hare is completely different. Hope this begins to shed some light on the subject!
By Lisa Shock on Tuesday, November 28, 2000 - 08:30 am:
Essentially, if you look at that chart, you'd think that the 1 or the 35 was the key to the Jill Main type if you only knew about the first 3 CDs, but, by looking at the other 3, you see that is irrelevant. (Plus lots of CDs have 1 track, and Don't make Jills! I also have several 1 track, 38 minute CDs that make Gaboos.)
By torey luvullo on Tuesday, November 28, 2000 - 06:05 pm:
i remember this conversation now. my 3 cds with pure jills did not fit the general pattern. i cant get to my daughters psx to measure the track length, but the 3 cds, no of tracks, and what they make in mr1 and mr2 are:
cd name & artist
tracks
mr1
mr2
king crimson/discipline
7
worm/dino
jill/jill
talking heads/naked
11
worm/golem
jill/jill
schnittke & luto/viola
6
plant/jell
jill/jill
makes for a muddled mess!
By Lisa Shock on Tuesday, November 28, 2000 - 08:37 pm:
What I have in my notes is 3 CDs from Torey, no titles:
Tracks
Minutes
Seconds
MR2
7
37
43
Jill/Jill
11
52
17
Jill/Jill
11
52
47
Jill/Pixie
By torey luvullo on Wednesday, November 29, 2000 - 05:32 am:
ok, the first is king crimson; the second is the talking heads. the schnittke luto cd i must have overlooked in our chat; that one runs well over an hour, as classical cds tend to do.
By torey luvullo on Thursday, November 30, 2000 - 05:13 am:
i remember long threads from b's board on this subject - a lot of people used to be very interested in this subject, and they all had their pet theories. however, if i read your study correctly, none of the following things have anything to do with how the mr games generate monsters off of cds:
1. the serial number 2. the title
and none of the following factors are the SOLE mechanism for generating monsters:
1. number of tracks 2. length of cd
now, what about the one track = mr1 pure effect? did you find anything more out about that?
By Richard Ryley on Thursday, November 30, 2000 - 06:11 pm:
Bumping in here for a moment. You seem to be saying that the monster itself is read from a table, based on the values read from the CD. That means that there are a finite number of monsters that CDs can generate. The stats and so forth are not read from data on the CD.
The number of monsters in the table, of course, could be very, very high. But that would mean that they are not truly random. (All monsters generated off CD were designed by the programmers, and probably balanced to prevent them from being too powerful)
By Lisa Shock on Thursday, November 30, 2000 - 07:55 pm:
You've pretty much got it.
I think the stats are also generated from a table in the game based on some part of the subcode, but I haven't been able to figure that out either. What I do know is that "Identical" CDs produce identical monsters with identical stats. Non-identical CDs may produce the same monster but the stats are different. The subcode data shown above is the entire set of data on that CD, there isn't any more. So, Main/Sub and stats are all derived from the numbers you see there.
I have spent most of my time trying to derive the chart for types, so have barely touched the surface on stats. To start, I'll need pairs of CDs that make all the same monsters in all 3 games, but the stats differ from each other. Then, the subcode data that differs should reveal where stats come from.
By Lisa Shock on Thursday, November 30, 2000 - 07:56 pm:
And, Foster's UPC number project is still a valuable idea, since that's the best data available to people while they are out shopping for CDs.
Unfortunately, this is waaaaay over my head. Glad to know that we have the minds here that can figure all this out!
By torey luvullo on Friday, December 1, 2000 - 05:18 pm:
as an entire unitary subject, you're right tony. but if you just concentrate on one aspect of it, like the jills, it is interesting [well, at least to me]. i had been thinking about cross linking my cd lists to see if i could detect any correlations, but lisa not only beat me to it, she dug a lot deeper than i planned to!
By Lisa Shock on Saturday, December 2, 2000 - 07:24 am:
And in response to Richard's comment "The number of monsters in the table, of course, could be very, very high. But that would mean that they are not truly random."
One indication that they are not random is the fact that every time you put one particular CD in, the same monster comes out every time, with the same stats.
By torey_luvullo on Saturday, December 2, 2000 - 09:10 pm:
well, now that you mention it, lisa...
that is usually true, but not always. everyone is familiar with the multiple pressings of mellow gold. however, i noticed that one of the cds posted by ABB4Life tonight [one of his less "metallic" ones ^_^] was neil diamonds "jonathan livingston seagull". he got a daton [tiger/pixie]. my copy of jls yielded a rock hound [tiger/golem]. i doubt that there have been too many pressings of jonathan livingston seagull...
similarly, my cd of the talking heads "speaking in tongues" yielded a gobi [golem/henger]. argos has posted that his gave him a dagon [golem/arrowhead]. it was a bit more popular than jls, but i wonder if there really are multiple pressings of speaking in tongues.
a review of the cd list will reveal several other discrepancies, i imagine. i wonder if they mean anything, and, if so, what it means...
By Lisa Shock on Saturday, December 2, 2000 - 09:22 pm:
Yes, you are correct. I meant to say that every time you use one particular copy of CD, (-Like my only copy of Backdraft, in my house, not the entirety of all CDs with the same title.) you always get the same monster with the same stats.
By torey_luvullo on Sunday, December 3, 2000 - 07:17 pm:
well, yes. since mr2 deals in percentages, is it significant that two different cds with the same title and similar other attributes form the same main, with different subs? in other words, do very similar cds come up with very similar monsters, or is it all over the map?
well, everyone,.... i just thought i would pop in here and say hello to everyone,. seeing as i have been away for sooo long,. i have been working sooo much,. that i put my playstation up for quite a while,. and i did drag it out again, with the release of Final Fantasy ix, and so then i have also gotten a bit back into Monster Ranching again,. hehehe,... and i did never really finish my rare cd experiment,. but i did kinda make a few conclusions,. but the findings i have, i really have no way of reading them,
i tried to make a list of the different rare cd's and list the track numbers, track times,. time on cd itself,. and tried to find the pattern. i realy dont know how the pattern exists, but it is indeed there,. i tried converting the times into binary code,. and hexadecimal code, etc,.. and well, the answer is in there somewhere,. but as i said, its hard to explain,.
if anyone is a technical person, or can find out how cd's are written,. let me know via email, please,.. we may be able to figure thsi out once and for all.
but aside from that,. how is everyone???? the radio station has had me super busy lately, which i guess is a good thing,.. oh, and for you episode fanatics,. i might even be getting a part time job at AD vision soon too,... so who knows,. i might be able to even truly be a Monster Rancher, heheh so wish me luck everyone!!!!!!
ok, this damn post is long enough, so i will talk to you all soon!!!!!!
By Lisa Shock on Monday, December 4, 2000 - 10:12 pm:
Good to see you again!
Try putting "subcode data" or "CD subcode" into a search engine for more info on what is on CDs. Also, you might have fun with CloneCD.
It was just really difficult because everytime I thought I had a pattern, Torey would tell me about a CD that made it invalid.
Good to see you again Foster!!! I think a lot of us are getting anxious for MR3 ( I know I am). Lisa posted a few screen shots somewhere on this quickly growing board.
By Richard Ryley on Thursday, December 7, 2000 - 05:14 pm:
You misunderstand, I didn't mean that the stats were generated randomly, I mean that the stats were read using "random" data read off the disk.
There are three possibilities here:
1) The data is read off the disk, and combined in some way into an index into a single table. One value could be read, or many. (In theory, what you've already discovered disputes that only one value is read) The table is extraordinarily large, but finite. That is, there are only a certain number of Suezos that can be generated, and if you somehow CDs that unlock them all, you will never find a Suezo with different stats than what you already have.
Note that this means that the same monster can be generated by different CDs.
2) Some data is read off the disk, and used to determine a type of monster off of the table. More data is then read, and a second table, of variations to the base stats for that type, is then determined. The stats are independent of the type, so while it is possible to get two CDs that produce the exact same monster, it is slightly less likely.
In this case, however, there would be patterns of stats that would show up on a regular basis. They would not be random with respect to each other.
3) Data is read from the disk. One or more values determine main, then other values determine sub, and still more are used to compute each of the stats, separately. The modifiers may or may not be read from a table but the important thing is that they are each independent.
Note that in this case, it is possible for a CD to produce the exact same monster as another CD, but with ONE stat different. More likely, of course, ALL stats will be different, and they will vary from CD to CD in a way that reveals no pattern.
The obvious problem with the last theory is that "rares" have to be an exception to it. The game has to first check for a rare CD it recognizes, and if it does, read the monster from a table. If it doesn't, it falls back on the "random" read of data from the CD to determine what monster to generate.
This brings up the possiblity that the data used to generate rares may NOT be the same as that used to generate stats. I think it likely that it is the serial number used to generate rares, as otherwise the programmers would have had to read the data from each of the CDs they wanted to have unlock a rare, and map it to their table.
Fortunately, you don't seem to be using rares in your tests, but if you were I'd say to ignore them. The tests you have done so far seem to imply a table to determine type, but whether further data is read to determine stats is not quite clear at this time.
By Foster on Thursday, December 7, 2000 - 08:58 pm:
ahhhhh,.. another good explanation,.... the posibility of a secondary table, if certain requirements are met,.
just wondering,. i think im gonna re-attempt something,. everyone knows about that pesky Beck/Moo combination, right? how there are i think 4+ different monsters that can be produced off of mellow gold?? anyone else notice that the Non-Moo monsters are all Mews?? i had a theory today, that it has to be total data that partly matches up.
Now, im not suggesting that the Mews have anything to do with Moo, i think thats just coincedence,. but,.. has anyone else noticed that different printings of the same discs, usually comes up with the same main type of monster, but with different sub-types??? Myself, i have a copy of fatboy slim's "You've come..." and instead of getting the rare, i got i believe a arrowhead/joker... i am gonna try looking into this aspect in the next few days.. hopefully it will also give some leads.
By Carl Carlisle on Friday, December 8, 2000 - 03:23 pm:
You most likely have checked this out, but just in case.
You've mentioned using blank disks ( that registered either a gaboo or unreadable )to determine their effect. 1.)For forensic uses, each company that produces blank disks, might have their own codes embedded as well. These codes might also be read by the Monster Rancher games. 2.)Reproductions of music and game cd.s might not always be using the blanks from the original companies. And, even if they are, it would be registered as having come from a different batch. POSSIBLY. I could be way off base about all this. 3.)The systems that burn or format these disks might also have their own registry codes as well.
Have you tried formatting and programming the exact same information on blanks that were manufactured from different companies? Maybe they'll produce different results. If they do produce different results, then the time and date may also have an effect.
By Lisa Shock on Friday, December 8, 2000 - 03:40 pm:
I don't have a burner....
But, your suggestion does explain a bit about different pressings. In general, when monsters vary from pressing to pressing, it's on music or CD-ROMs, not PSX Games. (Ok there are different versions of PSX games, like Doom II and Doom II 1.1, but you know the program on the CD is different.) Sony makes all US PSX games in one plant. They start with raw materials like plastic beads and aluminum sheets and use their system to make the CDs, so the "blanks" never vary.
By Carl Carlisle on Saturday, December 9, 2000 - 02:39 am:
What about the batch number? When I replaced my Tenchu game, it didn't produce that special Kato/???. Uh..If you already explained that, I didn't understand enough to catch it. This is new stuff to me.
By Lisa Shock on Saturday, December 9, 2000 - 07:27 am:
Well, with PSX games, they will fix glitches or add minor changes to games, like adding super-easy mode or a preview of another game. There's usually a difference in the box, or label on the CD. The biggest clue is often those "Greatest Hits" packages. Games that I know have changed are Doom II, Metal Gear Solid, and Resident Evil 2. One of the best places to see if there are different versions of a game is the Gameshark website, because they'll have different codes archived for them.
By Carl Carlisle on Saturday, December 9, 2000 - 01:42 pm:
Ah yes. My Tenchu ( and TECMO's Deception) has that greatest hits label. Have the people at the Gameshark company been of any help?
By Lisa Shock on Sunday, December 10, 2000 - 12:02 pm:
I haven't asked much at thier boards, but I presume that if there are a different set of codes for each release (for one model GS) the program itself was altered in some way.
By Morgan Schafer on Tuesday, February 6, 2001 - 02:22 pm:
Well, I may have missed this somewhere in the posts, but probably not. It occurs to me that if there are 392 monsters in MR2, and all of the ??? subbreeds are gotten off specific CDs... the remaining ones can be considered to be common monsters and therefore there is likely to be a set data structure within the game that the numbers play off of. So, given that a C++ data structure (yes, that's what PSX games are coded in) would be indexed beginning with 0, then monster number one (Pure Pixie I think) would be indexed by 0. Number 2 would be 1 and so on and so forth. Try using this when you compare sets of numbers, plus conversion into hexadecimal and octal (you never know). Also a reversed-indexing might be at play also. Or even a paired index of main and subbreeds. Also consider adding/multiplying sets of data from CDs... for example adding the minutes to the seconds and multiplying by 2 or something like that. It would take a lot of number crunching without asking the devlopment team directly.
By Varthlokkur on Wednesday, February 7, 2001 - 08:58 am:
Being a somewhat of a programmer (or at least I claim to be) and a big-time gamer, a couple of ideas popped into my head. First, I must say the number of tracks, track length or any other such obvious data would be the first thing I would avoid when creating the Shrine program. Why? Well, if it was that obvious, where would the mystery be? The Shrine program is much more sophisticated than that. Also, while reading this, please keep in mind, I am taking my approach to how I might script the Shrine program.
First off, we kick off the Shrine program and insert the CD. From a programmers stand point, I need to figure out the quickest and easiest way to read a code (the CD sub-code that Lisa talks about) and apply it to the monster. Let’s say that the CD code (what ever it is) is read into memory. What is the next step? More than likely, it checks the code against data hard coded into its program to determine if this is a special monster (i.e.: Kasumi). If so, it loads up that monster’s data either from the inserted CD or through parts of it’s own program. If it is not a special, it moves on to the next step, determining the monster type.
By reading the piece of code, the Shrine program checking a chart of 38 monster types to determine the base monster. Now, I would assume that there is a rarity amongst the breeds. I thing we see that in our CDs. Just look at the number of Hoppers and Jells versus Dragons or Mocks. And, there maybe more than two level thought commonly amongst games, there are three levels of rarity, common, uncommon, and rare. I haven’t really examined any CD list to take a good call on this. Now, this rarity is based upon a probability. Making another assumption , the programmers are reading a hexadecimal code of 4 digits which gives us a number 65,535 to deal with. If we base rarity on a simple base 16, we can assign rares a 1 in 16 probability (6.25%), uncommons a 4 in 16 (25%) and commons an 11 in 16 (68.75%). Now, we populate an array of 65,535 cells (single column) with the 38 base monster types keeping in mind the rarity. Note, this does not have to be a linear progression of population. As a matter of fact, that would be bad. The monster type should be scattered at around the array for better randomness.
Alright, so the Shrine program has now determined the monster type, now it is time to determine the monster sub-type. Sub-types are probably broken into rarities are well, probably rare, uncommon, and common. For this step, each monster type will have to have an array with all it’s sub-types (except those specials) scattered on it according to it’s rarity probablility (i.e.: if a monster has seven sub-types, four are common, two are uncommon and one is rare, then the array of 65,535 cells will have to have the information scattered appropriately). So, the Shrine program reads the next four digit hexadecimal number and picks the sub-type.
Now that we have the monster type and sub-type, the Shrine program can assign all the other pertinent pieces of data (stats, attacks, etc.). This is probably ground through an algorithm to limit the highs and lows of that data, checked, of course, by another array for that monster for correct info.
Finally, the Shrine program pops out our monster. Yay! And we are…confused? I sure am. And here is the kicker, just because I might have programmed it that way, does in no way mean that is how it is done. Programming is like walking through a thick dark forest without a map. You know where you are, where you have been and where you are going. But, how you get to where you are going could be will be completely different compared to the guy who is doing the same thing tomorrow. This makes figuring out the Shrine program vastly difficult.
Here is another thing to consider. Let’s say you read a code from a CD in hexadecimal. A70C 00CF 07D1 029E 9925 9DD0. Who is to say which number correlates to monster type, sub-type, or other data. Or, even worse, if the programmer decides to manipulate the sub-code in some manner (sin of two hexadecimals added together anyone?), then backwards programming of the Shrine program will be even more difficult.
I hope this helps you people on your quest. Maybe I will take a deeper look the whole problem. Unfortuantely, I do not have great amount of CDs. But, I think I have some duplicate monsters that I can do some side by side comparisons of.
But, first, I am going to lay down. I just gave myself a headache thinking about all of this. 8^)
Varthlokkur
By Lisa Shock on Wednesday, February 7, 2001 - 11:03 am:
Thank you so much for writing this reply! As I was working on this, it was so difficult for me to explain the complexity of the problem!
Try reading the work that Catsgodot has done, and posted at Josie Nutter's: here. The relevant threads are: "Of What From CDs", "Nirvana Four Times Over" "Here Comes Satan Claus" He has managed to take data I posted here, and duplicate the monster with entirely silent tracks on a burned CD. Now he's trying to derive the monster chart itself. He has his own website devoted to a project, you might want to take a look: Catsgodot's Page.
By Varthlokkur on Wednesday, February 7, 2001 - 12:15 pm:
Wow! He has done some great work. I'm going ot try to do some side-by-side analysis of some CDs I have that produce the same monster. Maybe with all this info around, we can figure this out.