New Mars Forums

Official discussion forum of The Mars Society and MarsNews.com

You are not logged in.

Announcement

Announcement: As a reader of NewMars forum, we have opportunities for you to assist with technical discussions in several initiatives underway. NewMars needs volunteers with appropriate education, skills, talent, motivation and generosity of spirit as a highly valued member. Write to newmarsmember * gmail.com to tell us about your ability's to help contribute to NewMars and become a registered member.

#26 2005-07-28 17:18:39

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

Many many apologies, I hope that wasn't going on all day. sad


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#27 2005-07-28 18:19:27

Palomar
Member
From: USA
Registered: 2002-05-30
Posts: 9,734

Re: New Message Board Software

Someone, or some thing, is doing massive searches on the forum. This has been causing the SQL timeouts today. I am turning off search capablity for Guest users. Sorry Guest users, but you will now have to register to do that.

*Is that what was causing New Mars to not open?  At first I thought it was my ISP (dial-up).  Rebooted the computer.  About an hour later while trying to access the board...it stalled.  Then a 3rd time, maybe an hour later.  Finally a 4th time, just before going offline for a couple of hours.  Window would say it's opening New Mars but it'd just drag very slowly and never open.  My ISP was fine and running briskly otherwise; different web pages opening quickly, etc.  It was just New Mars.

Glad you took that precaution, Josh. 

--Cindy


We all know those Venusians: Doing their hair in shock waves, smoking electrical coronas, wearing Van Allen belts and resting their tiny elbows on a Geiger counter...

--John Sladek (The New Apocrypha)

Offline

#28 2005-07-28 19:52:37

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

I went ahead and shortened the session lengths to (back to an hour), but the day is over with and I'll be applying the unread topics mod tonight, so no one should notice.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#29 2005-07-28 20:57:34

John Creighton
Member
From: Nova Scotia, Canada
Registered: 2001-09-04
Posts: 2,401
Website

Re: New Message Board Software

Someone, or some thing, is doing massive searches on the forum. This has been causing the SQL timeouts today. I am turning off search capablity for Guest users. Sorry Guest users, but you will now have to register to do that.

It could be a search engine. There probably is away to control what pages bots and spiders can see.

Offline

#30 2005-07-28 21:01:59

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

It's very possible, I'll put a robots.txt file up in the /forums directory.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#31 2005-07-29 05:01:45

Palomar
Member
From: USA
Registered: 2002-05-30
Posts: 9,734

Re: New Message Board Software

*Thought I'd mention:  "View posts since last visit" feature isn't working. 

--Cindy


We all know those Venusians: Doing their hair in shock waves, smoking electrical coronas, wearing Van Allen belts and resting their tiny elbows on a Geiger counter...

--John Sladek (The New Apocrypha)

Offline

#32 2005-07-29 06:16:06

Rxke
Member
From: Belgium
Registered: 2003-11-03
Posts: 3,667

Re: New Message Board Software

ditto; redirects to index page...

Offline

#33 2005-07-29 10:00:53

Palomar
Member
From: USA
Registered: 2002-05-30
Posts: 9,734

Re: New Message Board Software

*Hey guys.  The board continues to experience major problems.  Was inaccessible for nearly 2 hours.  This is the most recent error message:

phpBB : Critical Error

Error updating last visit time

DEBUG MODE

SQL Error : 1053 Server shutdown in progress

UPDATE phpbb_users SET user_session_time = 1122651999, user_session_page = 0, user_lastvisit = 1122648656 WHERE user_id = 197

Line : 187
File : sessions.php

Just thought I'd mention it.   :?  Now I'd better hurry and post those two articles; honestly, the boards are getting very unpredictable and unreliable. 

--Cindy


We all know those Venusians: Doing their hair in shock waves, smoking electrical coronas, wearing Van Allen belts and resting their tiny elbows on a Geiger counter...

--John Sladek (The New Apocrypha)

Offline

#34 2005-07-29 10:08:27

Cobra Commander
Member
From: The outskirts of Detroit.
Registered: 2002-04-09
Posts: 3,039

Re: New Message Board Software

There be demons here. Exorcism pending.


Build a man a fire and he's warm for a day. Set a man on fire and he's warm for the rest of his life.

Offline

#35 2005-07-29 13:18:19

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

Very bad news. NewMars has basically been crashing the SQL server due to, well, the traffic and posts that we have here. Now, it's almost a situation where we can laugh (it's a pretty impressive feat), but it's also a situation where we have to start questioning whether or not we stay with phpBB.

I might have to prune some forum posts. Probably will. Certainly full searching will have to be turned off ("view posts since last visit" and whatnot can stay on).

I'm in contact with out host and we'll work this out. But right now searching is going to have to go.

In the end it's probably my fault. smile


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#36 2005-07-29 13:28:03

Palomar
Member
From: USA
Registered: 2002-05-30
Posts: 9,734

Re: New Message Board Software

Very bad news. NewMars has basically been crashing the SQL server due to, well, the traffic and posts that we have here. Now, it's almost a situation where we can laugh (it's a pretty impressive feat), but it's also a situation where we have to start questioning whether or not we stay with phpBB.

I might have to prune some forum posts. Probably will. Certainly full searching will have to be turned off ("view posts since last visit" and whatnot can stay on).

I'm in contact with out host and we'll work this out. But right now searching is going to have to go.

In the end it's probably my fault. smile

*Josh:  There are a lot of old threads in Free Chat especially that you might want to consider "pruning."  Of course I know nothing about running a board like this, so it's just a suggestion.  There are old threads all around which maybe could use "pruning" as well.

You're suggesting we'd go back to the Ikonboard?  If so, it'll be back to the old name.  wink

BTW, it was 2-1/2 hours since my last post until I was able to access the boards again.  Not sure what others are experiencing.  Total of at least 4 hours today of inaccessibility.

--Cindy

P.S.:  I'm surprised you mention traffic being an issue in this.  We've had -less- registered users consistently online than from the Ikonboard, which seemed a lot busier.  Also, we've had less UNregistered "Guests" here on average as well.  Just thought I'd mention it.


We all know those Venusians: Doing their hair in shock waves, smoking electrical coronas, wearing Van Allen belts and resting their tiny elbows on a Geiger counter...

--John Sladek (The New Apocrypha)

Offline

#37 2005-07-29 13:38:10

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

Cindy, I've been struggling with the support people for about an hour and a half now, I pretty much got lucky here (catching it so soon and not working today). They shut down our database because of way too many queries. I take full blame for this, because I rebuilt the search tables and it doubled the size of our database in just a few minutes. No problems were occuring before I did that. It's totally my fault. And Im so sorry. sad

Going back to IkonBoard is not really something I want to do, despite phpBB being free, it is still pretty enterprise level. We just have a relatively cheapish host that needs to share its SQL databases with thousands of users.

If I have to I'll pay for a better host out of my own pocket.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#38 2005-07-29 13:38:41

clark
Member
Registered: 2001-09-20
Posts: 6,278

Re: New Message Board Software

Random idea...

Could the new forum be part of the problem? It is a new variable, right?

I imagine a search function on the Martian Chronicles forum returns a null value of some sort on searches conducted by non-members. Could that cause a loop of some sort by the bots?

edit- sounds like this isn't the issue. Keep us updated on the options.

Offline

#39 2005-07-29 13:44:35

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

The issue is that we had an 80MB search table that basically was queried, in full, any time someone did a search. For one host, this would be no problem, but we share our SQL database with many many users. Our database is probably one of the most significant ones out there.

This is all because I had to be smart and rebuild the search tables all the way back to 2001. Heh.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#40 2005-07-29 13:47:09

clark
Member
Registered: 2001-09-20
Posts: 6,278

Re: New Message Board Software

can you hack it to limit searches by year?

Offline

#41 2005-07-29 13:58:46

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

It's a possiblity, but I'm going to do some pruning anyway. The database problems should be fixed for good. I can't apologize enough for them.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#42 2005-07-29 14:01:46

clark
Member
Registered: 2001-09-20
Posts: 6,278

Re: New Message Board Software

What do you mean by 'prune'?

Offline

#43 2005-07-29 14:05:51

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

I mean that I'm going to delete posts, say, to this time in 2003. Maybe this time in 2004. I'll still have them, and we'll bring them back one day when we get a host that can handle them. But I'm waiting for a support response from our host before I do that. Things are looking okay from my end, but I want them to give me the "Everything looks okay."

edit: and if anyone by chance got a "password required" thingy just a minute ago, I was doing something stupid (but I realized my error within half a heartbeat, so the odds of someone seeing it are slim), I'm sorry guys, I'm having a bad time right now.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#44 2005-07-29 14:14:55

clark
Member
Registered: 2001-09-20
Posts: 6,278

Re: New Message Board Software

Not that many will care, but there goes some of the exsisting creative content on this board...

Offline

#45 2005-07-29 14:19:51

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

Nah, I'm going to delete threads, not posts. Special threads will remain. And as I said I'll still have all the old posts. Right now we just have to deal with SQL management stuff, which is something I never considered we'd have to do when we moved the forums. I mean, after all, SQL is better than any other database, right?


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#46 2005-07-29 14:21:23

Palomar
Member
From: USA
Registered: 2002-05-30
Posts: 9,734

Re: New Message Board Software

I'm sorry guys, I'm having a bad time right now.

*Josh, don't stress.  I hope I haven't sounded impatient or anything.  It's been a busy day with stat items which need immediate attention in my work, slow news and etc.  Sooooooo:  I wouldn't have been at the boards much today anyway.  My previous posts were simply keeping Mods/Admin appraised of the situation.

That last error message (the long one) did concern me, though:  I thought, oh no...don't tell me the entire database has crashed.  tongue  It didn't.  smile

You're doing a great job, giving it your best, and I'm sure I speak for everyone in thanking you...

--Cindy


We all know those Venusians: Doing their hair in shock waves, smoking electrical coronas, wearing Van Allen belts and resting their tiny elbows on a Geiger counter...

--John Sladek (The New Apocrypha)

Offline

#47 2005-07-29 17:27:06

John Creighton
Member
From: Nova Scotia, Canada
Registered: 2001-09-04
Posts: 2,401
Website

Re: New Message Board Software

I don’t know if anyone noticed but when the SQL went down the wiki was still up. I was thinking it is not so bad being down for a bit if people used that time to work on the wiki. However, if we do too many SQL queries we can’t take every other site down with us. So maybe we need to do some trimming as was suggested.

Offline

#48 2005-07-29 20:21:05

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

The forums and wiki have two independent SQL servers. Our host randomly assigns us an SQL host when you add a new database. I've never have it give me the same host twice. So that's why the wiki was fine but the forums were not.

Here's the response from support when I asked why the SQL db was locked:

Hello,

The database was locked to maintain the stability of the server.  It was driving up the load and causing it to crash.  It's approximately 192MB in size and running 362,000 rows at a time.  This had to be stopped.  Several of your tables need to be pruned, such as phpbb_search_wordmatch, and phpbb_posts_text.

If you have any more questions or need any further assistance, please feel free to contact us.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

#49 2005-07-30 08:54:30

Josh Cryer
Moderator
Registered: 2001-09-29
Posts: 3,830

Re: New Message Board Software

More info:

Hello Mr. Cryer,

Right now, the database looks good so I don't think any more pruning is
necessary. We will let you know if any further problems arise. Thank you
for tending to this issue and getting back to us.

Guest, I'll see if I can't get google to index us. At least once a day google bots are here (to a RDNS lookup on a given Guest IP and it's probably google). I don't have a robots.txt file up, I don't think google will search a forum unless explicitly asked to do so.

I'm looking at simply having the search tables only contain data from the last, say, 3-4 months. 4 years of posts is a lot for even MySQL to search.


Some useful links while MER are active. Offical site NASA TV JPL MER2004 Text feed
--------
The amount of solar radiation reaching the surface of the earth totals some 3.9 million exajoules a year.

Offline

Board footer

Powered by FluxBB