The great crash

SpaceNut · 2013-03-28 20:57:36

Hi Jim have you looked at the old apps page for the applications that would help with the older file formats....

http://www.oldapps.com/

There are developer tools available for the MySQL as well as the older php which I think were coverted as the revisions changed....

mariusz · 2015-01-30 07:21:08

Has this issue been resolved, or do you still need help recoving that 90GB of data?

JoshNH4H · 2015-01-30 08:18:14

I would say that about 50% of the data has been recovered

Rxke · 2015-01-30 08:20:30

This being a digital database: about 50%... You mean you guys recovered all the zeroes?

(sorry, I'll get my coat now)

JoshNH4H · 2015-01-30 08:22:22

haha I didn't recover anything... But I'd say there were about as many posts between 2008 and the crash as the Beginning and 2008

JoshNH4H · 2015-01-30 08:22:45

You should come by more, I haven't seen you around in ages

mariusz · 2015-01-30 09:18:40

JoshNH4H wrote:

I would say that about 50% of the data has been recovered

I believe I could help with restoring the other 50%. Who should I contact?

jburk · 2015-01-30 23:16:50

You can see some details of what is needed on the site itf.marssociety.org (bottom of page). I can arrange for you to get the mysql-bin files if you are interested in helping. Email me

mariusz · 2015-02-03 09:20:44

jburk wrote:

You can see some details of what is needed on the site itf.marssociety.org (bottom of page). I can arrange for you to get the mysql-bin files if you are interested in helping. Email me

Can you confirm that you've received my email? I emailed to both your email and webmaster@ but got no response.

Thanks

jburk · 2015-02-06 18:37:17

No I dont see any email from you. I sent you the files which are already hosted at a cloud provider. Please re-email me at either webmaster@marssociety.org or jburk@marssociety.org

Josh Cryer · 2015-02-19 01:53:01

If you guys manage to make this work out let me know and I can help guide / merge the databases together. I'm here infrequently these days but I still admit new members and act on moderator messages when they crop up. I know jburk asked me to help recover the data but I didn't feel comfortable handling such a large dataset at the time (and still not now, my knowledge level is quite amateur) but if the forum data itself is fixed just let me know via email and we'll figure out that part. If anything happens emailing me directly is the best bet though, if you email me with newmars in the title I get back within the next day. I'll look in to seeing how to do a merge in the meantime, if / when the data is ever recovered.

SpaceNut · 2015-02-19 21:10:45

I hope that they can be restored as well. Was searching around my hard drive and did locate some saved copies of forum pages from that era of topics that I had for topics of interest for future discusion.

We really had grown quite a bit back then with the changes in the topic forums and such.
one such page had these attributes with my post on page 5 of the topic and was 20 posts per page.

here is the info for page 1 ...newmars.com/forums/viewtopic.php?f=8&t=9014... and was in Board index » Acheron Labs » Life support systems at that time....
SpaceNut Post subject: Re: Adapting rapid prototyping concepts to habitat constructionPosted: Mon Dec 27, 2010 7:29 pm Pioneer Member Joined: Thu Jul 22, 2004 12:38 pm Posts: 6615 Location: New Hampshire Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group [ Time : 0.089s | 11 Queries | GZIP : Off ]

wow I just noticed that I had more posts back in 2010 than I have now...

Last edited by SpaceNut (2015-02-19 21:12:18)

SpaceNut · 2015-02-23 21:34:04

I have been trying some different things in the topics that have the pages shift nearly off screen.

* is a bracket that is removed

This appears

*color=#000000:post_uid0][/color:post_uid0*

to be a posting instruction that is no longer in the mysql data packet for posting...

post id 47122

removed this one from

*img:post_uid0][/img:post_uid0*

and this one

*quote][/quote:post_uid0*

post id: 73482

After removing these from the posts the pages shift back to normal position.
So the instruction in the bin lines have the old php forms data format....

SpaceNut · 2015-03-12 19:53:38

Also if the Admin's see this post have them remove these instructions from each post and we can fix a majority of the problems with the older posts.

[*color=#000000:post_uid0][*/color:post_uid0]
[*color=#000000:post_uid14][*/color:post_uid14]
[*i:post_uid0][*/i:post_uid0]
[*b:post_uid0][*/b:post_uid0]

jburk · 2015-03-14 00:58:07

SpaceNut, can you clarify? Are these some FluxBB codes that are being inserted into each post? How would we go about removing them from each of the older posts?

SpaceNut · 2015-03-14 09:21:39

These are part of the BBCode used to change the appearance of the post, making it appear in color, italic, bold ect.. that a user can place into the text that he or she wishes to post. I will use the coding to highlight the part of the string that is defective to make it red.

http://www.newmars.com/forums/help.php#bbcode

It's the :post_uid (sometime with a number) portion of the code that is no longer valid in the FluxBB bin string which when redisplayed from a corrupt bin the program does not know how to make it happen. These tags are visible in the corrupt topics that are showing the page shift issue.

SpaceNut · 2015-03-15 11:28:09

I have done an edit of this topic http://www.newmars.com/forums/viewtopic.php?id=2000
Where the following post had contained the error.

Once the final posts was edit'd the extra line in the display before the posting box disappeared
All other posts in the topic after that entry are uneffected as they were made by the new FLUXBB software.

Hopefully this will help...

SpaceNut · 2015-03-21 21:53:36

I was looking through the old topic posts that I saved on my harddrive which date beginning in Jan with some into Dec in 2010. These were the old phpBB Group forum software so the data that is in the crash have the changes that we had made in the forums, look on the home page as well as sub topic folders which held the forums topics.

These are just some of the formats.
'viewforum.php?f=4'
'viewtopic.php?f=1&t=5226'

I was also looking at the first mysql-bin.000004 file in notepad first line of code is (þbin-QJ ^ b 5.0.51a-log) and I can see as well as read the posts that were made in it by me and dunwich as well as others contained in it. The topic names for post entries were Spammer, Americium power, NASA may abandon plans for moon base (let\'s make it happen) I could see the topic ID and post count for the entry as well.

I was able to see the post made by Josh Cryer id = 33 which is now 34 in New Message Board Software topic_id = 3694 post_id = 135522 early in the file...my id was also incremented by 1 as well.

It would be possible to create a text file from these bin's with info of poster, date, topic name, content and other details for getting them back but it would take time.

Last edited by SpaceNut (2015-03-21 21:57:18)

SpaceNut · 2015-03-25 20:40:37

I have begun to create a sitemap as such to allow for me to be able to determine what might have been reused once the forum was restarted to make use of for extracting the content of the bins....
Will keep you all posted as I go as to what is recovered...

SpaceNut · 2015-03-28 21:03:39

What is thought to be just data for the forum in the bin is in actuality a log file with all sessions activity.

The opening line in the logs indicate that the MYSQL version at the time was 5.0.51a .... (session_user_id, session_start, session_last_visit, session_time, session_browser, session_forwarded_for, session_ip, session_autologin, session_admin, session_viewonline, session_page, session_forum_id, session_id) are the columns in the table each activity is logged into the database and outputs into this file format.

jburk · 2015-03-31 00:32:33

Yep, the idea is that if we can recreate the schema of the database at that time, we can "replay" back the logs in the mysql-bin.* files and get a revived database. Then, from that revived database, we can prep an import SQL file to restore the missing posts in this forum.

If you can come up with the exact schema including datatypes I can set up the database somewhere on the newmars.com domain.

SpaceNut · 2015-04-04 21:48:48

I think that I may have found why the data base did crash in one of the files...

"user warning: Table './drupal/cache_page' is marked as crashed and should be repaired\nquery: UPDATE cache_page SET data = '<br />\\n<b>Warning</b>: Cannot modify header information - headers already sent by (output started at /data/www/drupal/includes/database.mysql.inc:128)

I also have what seem to be the wiki pages so I am thinking that the 2 sections that we had at the time were using the same data base but by different software.....

The internal link for the Smart 1 page was http://www.newmars.com/forums/viewtopic.php?t=1591 which is no a bad link after the crash which was located in wiki/index.php/Electric_propulsion\ at the time.

jburk · 2015-04-04 22:40:29

I would be interested in recreating the wiki, if nothing else for archival purposes, so if you can extract out those articles it would be useful.

SpaceNut · 2015-05-02 17:13:52

I tried to make this post to topic : Alternative Space ventures - are we on the road to cheaper access http://www.newmars.com/forums//viewtopi … d=1128&p=4
but recieved an error:
Info
Bad HTTP_REFERER. You were referred to this page from an unauthorized source. If the problem persists please make sure that 'Base URL' is correctly set in Admin/Options and that you are visiting the forum by navigating to that URL. More information regarding the referrer check can be found in the FluxBB documentation.

WoW never seen that before.....

Here is the content that I was going to post below:.....
Flat FAA AST Budget Could Slow Growth for Commercial Space Industry

This week the House Appropriations’ Transportation, Housing and Urban Development subcommittee passed their Fiscal Year (FY) 2016 spending bill. The legislation includes funding for the Federal Aviation Administration’s (FAA) Office of Commercial Space Transportation (AST), which is a part of the Department of Transportation. The bill did not approve FAA AST’s $1.5 million requested budget increase for FY 2016, keeping FAA AST’s budget flat relative to their FY 2015 budget.

Just this week, Blue Origin conducted a successful development flight of their New Shepard launch vehicle, and SpaceX conducted its fifth operational launch in four months. To continue on the current trajectory of the sector, we must ensure that the FAA AST has the resources it needs to work with the industry in a manner that will continue to promote growth, and improve public and occupant safety. The commercial space industry and government are partners in the economic development of space, and CSF looks forward to working with Congress to support this increase as it improves this bill through the legislative process.”

SpaceNut · 2015-05-02 21:42:02

Here is another oddity that I found in the sig line post 39 in topic http://www.newmars.com/forums/viewtopic.php?id=1128&p=2

Give someone a sufficient [b:sig_uid][i:sig_uid]why[/i:sig_uid][/b:sig_uid] and they can endure just about any [b:sig_uid][i:sig_uid]how[/i:sig_uid][/b:sig_uid]

Also I fixed all the previously meantion code on page 1 and 2 and the topic is now allowing posts to it. See my post #38 for what was an issue....

Last edited by SpaceNut (2015-05-02 21:49:54)

New Mars Forums

Announcement

#26 2013-03-28 20:57:36

Re: The great crash

#27 2015-01-30 07:21:08

Re: The great crash

#28 2015-01-30 08:18:14

Re: The great crash

#29 2015-01-30 08:20:30

Re: The great crash

#30 2015-01-30 08:22:22

Re: The great crash

#31 2015-01-30 08:22:45

Re: The great crash

#32 2015-01-30 09:18:40

Re: The great crash

#33 2015-01-30 23:16:50

Re: The great crash

#34 2015-02-03 09:20:44

Re: The great crash

#35 2015-02-06 18:37:17

Re: The great crash

#36 2015-02-19 01:53:01

Re: The great crash

#37 2015-02-19 21:10:45

Re: The great crash

#38 2015-02-23 21:34:04

Re: The great crash

#39 2015-03-12 19:53:38

Re: The great crash

#40 2015-03-14 00:58:07

Re: The great crash

#41 2015-03-14 09:21:39

Re: The great crash

#42 2015-03-15 11:28:09

Re: The great crash

#43 2015-03-21 21:53:36

Re: The great crash

#44 2015-03-25 20:40:37

Re: The great crash

#45 2015-03-28 21:03:39

Re: The great crash

#46 2015-03-31 00:32:33

Re: The great crash

#47 2015-04-04 21:48:48

Re: The great crash

#48 2015-04-04 22:40:29

Re: The great crash

#49 2015-05-02 17:13:52

Re: The great crash

#50 2015-05-02 21:42:02

Re: The great crash

Board footer