The great crash

SpaceNut · 2020-12-15 18:51:15

As we have closed the forum off to spammers, I wanted to see when the crash happened by looking at the members list ascending

Username Title Posts Registered
target tony Member 2 2008-05-27 profile? id=3030
Decimator Member 39 2011-11-20 profile? id=3035

This gap means that we have other gaps such as in the topic id numbers as well...

tahanson43206 · 2020-12-15 19:59:18

For SpaceNut re showing topic...

Thanks for bringing this topic back into view ...

jburk wrote:

As part of modernizing & building out Marspedia.org:
I have restored the New Mars Wiki! - wiki.newmars.com
Right now, creating new user accounts are turned off to prevent spam, but let me know if you want me to set one up for you.
Enjoy!

I finally took the opportunity to visit the Mars Wiki ...

There appears to be a lot there to study.

It might be a place we could consider for "permanent" storage of some of the wisdom that flows through this forum.

(th)

kbd512 · 2020-12-16 16:56:18

SpaceNut or tahanson43206,

What is it that needs to be done to fix all the messed up topics?

Is this a simple matter of writing a JavaScript or PHP script to fix all of them?

SpaceNut · 2020-12-16 17:16:41

the invalid parts that shift plus commands are from the old conversion in the files contained in the mysql data base.
attributes to quoting of posts , color, underlining of text, making text bold, giving the text an italic appearance ect...

post 38/39 give the structure of the command where the (:post_uid##) is the ones that will cause the shifting of the posts in the topic.
post 73 has the topic reference conversion error for links within the forum
post 5 gives the third error type in its post...
post 33 last issue is the sub forums which where created in the data gap that James has recovered
This is current userid name http://newmars.com/forums/profile.php?id=###
but I have seen Pid=## somewhere and not all of the ##' are sequential...
The same sequential ## system for the topics also have gaps in the number as well as the post counters....

since that data is on the server files you need direct access from some one like the webmaster James has...

kbd512 · 2020-12-16 18:29:44

SpaceNut,

Is there any reason you can't go through the posts with a script to correct / replace the offending tags?

SpaceNut · 2020-12-16 18:42:21

Since the only way to fix a post is in edit as the person that wrote it to fix the defects.
If you are the moderator you can also fix others along with your own but you can not fix any admin posts.
It takes being an admin with edit to be able to fix them.

here is a sample quote post from a defective topic with a ( before the color to remove the [ that would normally be there..
In original posts with quotes it will contain the :post_uid stuff when its shifting.

Gennaro wrote:

(color=#000000:post_uid12]I was planning to write a new message to the the thread "Commercial Viability of Space", initiated by me some weeks ago. Prior to start writing, I came upon a few minor details in need of editing in the original post (spelling errors and the like; items I thought were already cleared in fact), but as I pressed the "submit modified" button, most of the messages disappeared.
The symptoms seem identical to what happened to "Who needs a Space Station?", also started by me.
What's going on here? Can it be rescued? I'm very sorry about all this, the mess etc, and additionally so since it was probably one of the better posts I have written on this forum and there were a number of interesting comments I would have liked to pick up from.
(/color:post_uid12]

You would need to open every post multiple times to make the edits for all of the defect types that a single post might contain since the number can be different. The color is also something that no longer is needed to make the post distinguishable to others plus color numbers are not correct...

kbd512 · 2020-12-16 18:51:19

SpaceNut,

That metadata must be stored somewhere. Why are you trying to hand-correct something like that? Even if we do have to do it from the front end, why can't we use a browser automation tool and a macro script that contains a list of regular expressions to search for and replace within the posts that have been messed up?

tahanson43206 · 2020-12-16 18:54:12

For kbd512 re post recovery ....

It is possible you may see a way to automate the repairs SpaceNut has been doing.

In any case, thanks for your interest in helping to bring some of those old topics back to life!

I was taken aback a few minutes ago, to read the post from Clark about the forum actually starting back in 1995 or so!

I thought 20 years of history was impressive! 25 years would be even more so.

In one of the topics SpaceNut recently brought back into view, I read a post from James Burk about the nature of the problem caused by a failed hard drive. As I remember the post, he was able to save the log files from the MySQL database, but could not recover the schema.

It might be possible to analyze the log files and write code to re-generate a database with a structure to match the current one, using the log files as input. That would be some heavy duty coding, but (I'm assuming) not out of the question.

I've been asked to so something similar with another space related web site, and I'm hoping to schedule time to work on that over the winter.

(th)

SpaceNut · 2020-12-16 18:59:44

Its in the MySQL files which get ported to our display.. we do not have direct access to it....

That said a script must pull up each post to fix the defects...

here is the topic that has the post which I identified
http://newmars.com/forums/viewtopic.php?id=3526

as you see the topic progressively get shifted the further down you go....
highlight a word and use the tab key to move the screen so that the function buttons are visible for editing....

SpaceNut · 2020-12-16 19:15:04

Here is a copy of a piece of the log file

þbinXCLI ^ b 5.0.27-log XCLI8
K kCLI g @ ` std! ! ! newmars_forums UPDATE phpbb_sessions SET session_time = 1229734762, session_page = 'viewtopic.php?f=5&t=4529&start=720', session_forum_id = 5
WHERE session_id = '61c2c3f08d03f262b2814785b5b4eb4e'mCLI € ç @ ` std! ! ! newmars_forums DELETE FROM phpbb_sessions WHERE session_user_id = 3078mCLI + @ ` std! ! ! newmars_forums INSERT INTO phpbb_sessions (session_user_id, session_start, session_last_visit, session_time, session_browser, session_forwarded_for, session_ip, session_autologin, session_admin, session_viewonline, session_page, session_forum_id, session_id) VALUES (3078, 1229734765, 1229734765, 1229734765, 'Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)', '', '67.195.37.107', 0, 0, 1, 'viewtopic.php?f=1&t=3803', 1, '09957d99a15325d180f9b0aee5f0f806')mCLI ˜ ª @ ` std! ! ! newmars_forums UPDATE phpbb_users
SET user_lastvisit = 1229734765
WHERE user_id = 3078nCLI » e @ ` std! ! ! newmars_forums UPDATE phpbb_topics

I was able to pull from it posts activity and ore from them by hand but its time consuming....

SpaceNut · 2020-12-16 19:20:35

here is a piece that contains a post
had to remove the e in the brackets for quote

WHERE session_id = 'fa14ddb7761b084a891a3369db5210cb'DLI S* sÜ DLI Ë . # @ ` std! ! ! newmars_forums INSERT INTO phpbb_posts (forum_id, poster_id, icon_id, poster_ip, post_time, post_approved, enable_bbcode, enable_smilies, enable_magic_url, enable_sig, post_username, post_subject, post_text, post_checksum, post_attachment, bbcode_bitfield, bbcode_uid, post_postcount, post_edit_locked, topic_id) VALUES (13, 6125, 0, '24.17.123.23', 1229734918, 1, 1, 1, 1, 1, '', 'Re: Obama not a US citizen?', '[quot="bobunf":1ogajlh8]S_Lajoie wrote, "he had his Hawaiian birth records sealed, so no one could get them. He also had his Kenyan birth records sealed."\n\nWhat are your sources for these assertions?\n\nBob[/quot:1ogajlh8]\n<a class=\"postlink\" href=\"http://www.worldnetdaily.com/index.php?fa=PAGE.view&pageId=79174\">http://www.worldnetdaily.com/index.php? ... geId=79174</a>', '571034e9d4948f29914b0568217c0781', 0, 'gA==', '1ogajlh8', 1, 0, 6565)DLI œ º. # @ ` std! ! ! newmars_forums UPDATE phpbb_config

kbd512 · 2020-12-16 22:14:43

SpaceNut,

Here's a regular expression string that can strip quotes:

\[quot.+?\]|\[\/quot.+?\]

Here's what it matches:

[quot="bobunf":1ogajlh8]

[/quot:1ogajlh8]

We would obviously perform more exhaustive testing to be sure that what it matches is what we actually want to remove, but it can be done. If there's a pattern to these offending character strings, then we can probably text match and strip them using a combination of scripts and regular expressions. Heck, this can even be done from MySQL. I use regular expressions to text match data stored on various databases at work, every so often. Even if we have to do it from the front end, there are free tools to automate Firefox, so we can have the macro scan through each page of posts, sequentially, and remove the defunct tags.

Edit:

The link below has the tool I used to test that expression.

https://regex101.com

Last edited by kbd512 (2020-12-16 22:15:42)

SpaceNut · 2020-12-17 16:52:21

Here is a section that has the web page that was done with DRUPAL

@ std! ! drupal UPDATE cache_page SET data = '<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\r\n\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\" lang=\"en\" xml:lang=\"en\">\r\n \r\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />\r\n\r\n <title>FMARS 2009 | Flashline Mars Arctic Research Station</title>\r\n \r\n <meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />\n<link rel=\"EditURI\" type=\"application/rsd+xml\" title=\"RSD\" href=\"http://www.fmars2009.org/blogapi/rsd\" />\n\n<link rel=\"alternate\" type=\"application/rss+xml\" title=\"FMARS 2009 RSS\" href=\"http://www.fmars2009.org/rss.xml\" />\n\n<link rel=\"shortcut icon\" href=\"/misc/favicon.ico\" type=\"image/x-icon\" />\n <link type=\"text/css\" rel=\"stylesheet\" media=\"all\" href=\"/modules/aggregator/aggregator.css?B\" />\n<link type=\"text/css\"

This is the page content

<div class=\"node\" ><div class=\"node-inner\">\r\n\r\n \r\n <div class=\"teaser\">\r\n \r\n <div class=\"title-wrapper clear-block\">\r\n \r\n <div class=\"teaser-title\">\r\n <div class=\"wrapper\">\r\n <h2>\r\n <a href=\"/content/flashline-mars-arctic-research-station-2009\">Flashline Mars Arctic Research Station 2009</a>\r\n </h2>\r\n </div> \r\n </div>\r\n \r\n <div class=\"teaser-content\">\r\n <div class=\"wrapper\">\r\n <h3>Project Overview</h3>\n<p>In order to help develop key knowledge needed to prepare for human Mars exploration, and to inspire the public by making real the vision of human exploration of Mars, the Mars Society maintains multiple habitats around the world, where simulation missions can take place in conditions as similar as possible to actual Mars missions. The Flashline Mars Arctic Research Station, brought online in 2000, was the first station created in support of this goal.</p>\n<h3>

This is also in the log files which were all that was in the bin after the crash.

At that time it was 90 gig of log files

kbd512 · 2020-12-17 19:19:58

SpaceNut,

So, what do you need done with all of those tags? Do they need to be removed? If so, which ones?

SpaceNut · 2020-12-17 19:35:19

The conversion issue topics are the example in post 84 for the tags which still remain here in the posts from the beginning to about 2005 entries from what I remember.

I had been doing the fixes as I found them but one could do so much more trolling for the identifiers that cause it with a script.

Try the script repair on that first topic if it works keep going to others....

The drupal is no longer an issue as they have there own server now for the FMAR website and are no longer sharing...

kbd512 · 2020-12-17 19:38:19

SpaceNut,

If we want to convert specific tags to retain formatting, then we can do a search and replace with the regular expressions.

SpaceNut · 2020-12-17 19:51:55

this topic has the on close inspection of posts the bold and italic with the tag extensions that cause normal display
http://newmars.com/forums/viewtopic.php?id=3585

of which we would want to retain the corrected appearance...

this one has the quote tag issue that causes the pane to shift
http://newmars.com/forums/viewtopic.php?id=3583

New Mars Forums

Announcement

#76 2020-12-15 18:51:15

Re: The great crash

#77 2020-12-15 19:59:18

Re: The great crash

#78 2020-12-16 16:56:18

Re: The great crash

#79 2020-12-16 17:16:41

Re: The great crash

#80 2020-12-16 18:29:44

Re: The great crash

#81 2020-12-16 18:42:21

Re: The great crash

#82 2020-12-16 18:51:19

Re: The great crash

#83 2020-12-16 18:54:12

Re: The great crash

#84 2020-12-16 18:59:44

Re: The great crash

#85 2020-12-16 19:15:04

Re: The great crash

#86 2020-12-16 19:20:35

Re: The great crash

#87 2020-12-16 22:14:43

Re: The great crash

#88 2020-12-17 16:52:21

Re: The great crash

#89 2020-12-17 19:19:58

Re: The great crash

#90 2020-12-17 19:35:19

Re: The great crash

#91 2020-12-17 19:38:19

Re: The great crash

#92 2020-12-17 19:51:55

Re: The great crash

Board footer