Skip to Content

Executing second phase of database upgrade

Vyxe's picture
Sun, 06/10/2012 - 08:45 — Vyxe

Going to see if I can keep this site online during it.... but we'll see.  This is expected to last through most of today (you can imagine the amount of data in the table going back to 2009 for every day for every player in the mylotro leaderboards.. whether they played or not)

No votes yet

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Vyxe's picture

Re: Executing second phase of database upgrade

Vyxe - 06/15/2012 - 22:30

Well.. here's my DOH moment... I copied everything over.... and had my default engine still being myisam... so... going to reconvert the thing again... there goes a week hahaha... the good thing is that I had copied the entire table (all 10gb, 51 million records worth)... so i'm going to be just altering that table to innodb and keeping the other one online to support this website.

Your rating: None
Vyxe's picture

Re: Executing second phase of database upgrade

Vyxe - 06/15/2012 - 15:27

Cranking through 2012 Snowbourn... so... at ~1 hour per world, I might be able to transfer to the new table tonight!  Maybe...

Your rating: None
Vyxe's picture

Re: Executing second phase of database upgrade

Vyxe - 06/13/2012 - 22:50

Whew.. 2011 migrated.  Next stage is Jan-May 2012 (this will let me do a really fast migration of June data which is what's being updated currently).  Can't start tonight as the script starts scraping in just a few hours (which locks the tables and will crash everything), so... eventually i'll get this thing migrated over and this site operating off of the new table!

I hope my research and theory that this will increase site stability and speed is right :)

Your rating: None
Vyxe's picture

Re: Executing second phase of database upgrade

Vyxe - 06/11/2012 - 22:37

So far up to Meneldor (again only 2011 data).  So this site will continue to be a bit slow while I transfer tables over.  When it's complete, I will cut this site over to use the new tables... which... theoretically will be faster and more stable.

Your rating: None
Vyxe's picture

Re: Executing second phase of database upgrade

Vyxe - 06/11/2012 - 07:21

doh.. permission problem on new server... didn't automatically run the scraping script.  Firing it up now.

Your rating: None
Vyxe's picture

Re: Executing second phase of database upgrade

Vyxe - 06/10/2012 - 21:19

Hah, yes, just for 2011 the Elendilmir set was about 2.5 million records.  I am converting from myisam to innodb because one of the major things killing this website are table locks.  This work also lets me set a couple more keys.

So far i'm through Firefoot.

Your rating: None

Re: Executing second phase of database upgrade

Player - 06/10/2012 - 21:02

Huge work ur doing, I know what this tablework means...
Further good luck and much fun.

greetz, Thoggar

Your rating: None
Vyxe's picture

Re: Executing second phase of database upgrade

Vyxe - 06/10/2012 - 16:28

@1630 MT Sunday, Current status - The following worlds are complete for uploading in the new table in terms of 2011 data (2009-2010 already uploaded for all)

Anduin

Arkenstone

Belegaer

Brandywine

Crickhollow

Dwarrowdelf

Eldar

Your rating: None

Post new comment

CAPTCHA
This question is for stopping spam bots. Logging in removes captcha requirements
  _____  ____    _              __  __  _ 
 |__  / / ___|  | |__     __ _  \ \/ / (_)
   / /  \___ \  | '_ \   / _` |  \  /  | |
  / /_   ___) | | |_) | | (_| |  /  \  | |
 /____| |____/  |_.__/   \__, | /_/\_\ |_|
                            |_|
Enter the code depicted in ASCII art style.


Dr. Radut | blog