Housekeeping

SpaceNut · 2021-05-05 17:48:49

Place duplicates with the incorrect email into the idA's and restored the remaining from 4737 through to 4791 and these are in the user table and ready for use.

tahanson43206 · 2021-05-05 17:55:06

For SpaceNut ...

Following up on the mystery of Bans table duplicates ....

Here is a quote from the web site where the FluxBB antecedent tables are stored:

bans
Field Type Default Description
id    int    The auto-incrementing (identity) primary key identifier for this table.
username    varchar(200)    NULL    The username this ban applies to.
ip    varchar(255)    NULL    The IP address(es) or partial IP address(es) this ban applies to.
email    varchar(50)    NULL    The e-mail address or e-mail address domain name this ban applies to.
message    varchar(255)    NULL    A message that is displayed for the banned user.
expire    int    NULL    A UNIX timestamp representing the day when the ban expires.
ban_creator    int    0    An ID number of user who create the ban.

Amazingly, the ID of the User Record is ** NOT ** included in the table!

The only connection to the original record appears to be via the Username field.

I ** assume ** that when the name of the primary record for a user is changed, FluxBB makes a good faith effort to find the corresponding Username in the Bans table ... I would imagine (as a first guess) that FluxBB grabs the first record it finds and changes the Username of ** that ** record. It would be interesting to know if it goes beyond that.

However, in any case, when you unban a user, the unban would presumably only reach the Bans table when there is a match on the user name.

If you make any discoveries about the behavior of the system with respect to the Bans table, I'll definitely be interested.

It is also possible to study the php code, but I'm hoping we can get past this situation without having to do that.

Eventually the only records in the Bans table should be those people with posts you want to keep available for future forum readers to find, in the context of the topic to which they contributed.

(th)

SpaceNut · 2021-05-05 18:01:14

ZacharyAccen is a spammer account reset to zero posts to convert it...same for USZEliza70,,,,same for HarriettSt...HershelBro also is a spammer reset to zero with the final FelipaRomo also a spammer with posts now set to zero all can be converted....

tahanson43206 · 2021-05-05 18:13:39

For SpaceNut re #1103

Thank you for looking at those ID's ... It's extra work for you, but we have plenty of time to get this right.

I'll pick up again at the tail end of the query, but now we know that one page query operations are workable if you are at work or otherwise not available.

***
I was happy to see the link to the tail end of today's SN15 flight ... someone was kind enough to start the video at 6:08, so I got to see the ship standing tall and burning off some methane without apparently causing a problem for itself.

Edit at 21:31 local time ... Yucaliptous2 had a post

I'll return to one page runs until you get a chance to look at it.

(th)

Last edited by tahanson43206 (2021-05-05 19:32:01)

tahanson43206 · 2021-05-06 06:20:47

For SpaceNut ... successful run in the "BA" group

It was only for 10 ID's ... there were two date exceptions ... no stops

Total Command Lines found: 123
Total input Lines in script: 386
Number of ID's processed: 12
Starting Number: 4809
Last Number of Run: 4820
Summary for Web Automation Report for 05-06-2021 at 08:14:05
Average time of Loop: 00:03:19
Total time of Run: 08:50:02
2 Date Exceptions were recorded.

I'll stay with BA's and try to run them out ...

The BA group has reached 4839 so far, and is still going.

Edit at 14:11 local time:

BorisS4052 has a post

I'll move on to the next group ....

CarolEdila has a post

I'll move on to the next group ...

Cesaryxpob has 6 posts ... I'll move on to the next group

cowUnwink has 4 posts ... I'll move on to the next group

DavidHip ... this is interesting ... the Username is in the Bans table, but not in the User table

I'll move on to the next group

DeweyMcLan had 1 post ... I'll move on to the next group

divinecosmetic had a post ... I'll move on to the next group

dorpornxxxwab had one post ... I'll move on to the next group

fawevkur has a post ... this may be a duplicate ... I'll move on to the next group

Gaunormm is another name in the Bans table that does not exist in the User table.

Will try the next group ...

getusamap had a post ... by coincidence, it was banned by kbd512. I don't see a lot of those.

On to the next group ...

GitaCrowde has a post ... on to the next group

Gordyj86 has three posts ... on to the next group

Gubbins has at least one post ... Calling it a day.

Best wishes to everyone for a productive Friday, whatever you're doing.

(th)

Last edited by tahanson43206 (2021-05-06 20:13:05)

kbd512 · 2021-05-06 17:35:02

Now that SpaceNut and GW have had computer problems, I'm having computer problems. I need to either buy a new MacBook or replace some more hardware on this 10 year old model. I've already replaced the hard drive and RAM, so I'm not sure how much life this little engine has left in it, although if I can find a parts bin replacement, I kinda want to see how long I can keep running what I have. It's either a display or graphics chip issue.

tahanson43206 · 2021-05-06 17:44:15

For kbd512 re #1106

My recollection is you reported you are an Apple owner. A relative is a (now retired) teacher who went with Apple when the school district did, and he's been an enthusiastic Apple customer ever since. He's kept the older systems around because (he says) the software on them suits him better than the newer version, but he got into the habit of backing up everything six ways to Sunday because when an Apple ** did ** fail for him, he lost a ** lot ** of data.

My guess is that you might well find that other Mac owners are equally reluctant to part with older equipment, but there might be one or two offered on eBay. If that's the case, and you can confirm the equipment is still working, then it should be possible to pop your hard drive into the eBay machine and go some more!

(th)

SpaceNut · 2021-05-06 21:27:18

I could swear I did a post on the outcome of the days run of ids which was clean of email duplicates and ran from 4792 through to 4930 which have been all unbanned and moved to the user list.

Here is the run down on the other users you were questioning

Yucaliptous2 has the Epilepsy topic of which drugs not on topic were mentioned

BorisS4052 travel agent spammer now reset to zero
CarolEdila spammer reset to zero

Cesaryxpob this appears to be a bot since the number of characters are like a twitter post which makes the user suspect
cowUnwink short post but location is afghanistan which makes the user suspect

DavidHip typo's I think it was DavidBop just did not post

DeweyMcLan spammer reset to zero
divinecosmetic spammer reset to zero
dorpornxxxwab email extension ru so spammer reset to zero
fawevkur spammer reset to zero

Gaunormm not sure what this is as the name does not come up

getusamap spammer reset to zero
GitaCrowde spammer reset to zero
Gordyj86 spammer reset to zero
Gubbins deleted toilet spammer post and reset to zero

kbd512 · 2021-05-06 23:39:35

tahanson43206,

For whatever reason, throwing away machines that can still be used really irks me. I guess it shouldn't, seeing as how we live in the land of plenty, but it still does. Maybe I inherited that from my depression-era grandparents. Frugality must have skipped a generation or something. If I had my druthers, I'd still be driving my 2005 Impala or my 2012 Silverado before the first one was wrecked (not by me) and the second one was stolen. I fail to see the point of buying new things whenever existing things will get the job done. The shirt I'm wearing right now has a few small holes in it, but I'm working from home and nobody cares, so it's going to stay in service until it literally falls apart. It's a few years older than the laptop is. I've had some stuff for so long I forgot when exactly I bought it.

If I can't find replacement parts for my 2010 / 2011 vintage MacBook Pro, then I guess it's time for a new MacBook. I kinda want to find out if the new Apple Silicon is everything that it's supposed to be. We bought ye olde MacBook Pro used from someone who had to have the latest and greatest, so maybe I'll see if I can find a gently used replacement on eBay that will last another 10 years or so. Anyway, this machine has been a real trooper. It's survived some less-than-gentle handling while I was still a consultant flying around the country every week. Sooner or later, all things must come to an end. That's just part of the human condition.

tahanson43206 · 2021-05-07 05:25:35

For kbd512 re vintage MacBook Pro .... I'll be interested in how things turn out ...

I'm a member of a tech group that talks about Linux as the primary topic, but everyone (apparently) owns Macs (except me). They spent quite a bit of time looking at the online specs for the latest Mac desk model, which uses an Apple chip, instead of one of the three major chip brands as previously.

We're in housekeeping so I'll toss this in .... I saw announcements recently that IBM has developed a 2 nanometer chip architecture. If anyone in the forum finds out more about that development, please put a link up in the Technology Updates topic.

***
For SpaceNut ... thanks for taking the time to look at all those "discoveries" in the Bans table.

I'm thinking about a major upgrade for WBA ... at present, it is designed around a read-query-from-top model, which halts when a problem shows up with an ID. I'm working around that for now by breaking the population into segments that will fit on one page at delivery time. The advantage is that I can count on the same number of tabs to reach the ID of interest, whereas the larger query has a variable number of pages and therefore a variable number of tabs.

The program is NOT currently smart enough to adjust to changing page count.

An upgrade would involve reading the primary query separately and collecting all the Usernames into an array, and then processing from the array.

That method would permit more adaptive handling of the occasional ID with posts, many of which contain content of value that needs to be preserved.

In the mean time, I'll continue chipping away at the one page queries.

(th)

tahanson43206 · 2021-05-07 06:06:30

For SpaceNut .... WBA is looking for ID's with posts these days ... The Last Visit test is set to 130 days, and even for longer gaps i've been taking your advice to upgrade them, since there were no posts.

The All Id's search didn't get past the first ID ....

Yucaliptous2 has a post ... I'll resume with the smaller groups

Edit at 15:42 ... Volunteer day here so WBA gets to take a rest.

I'm starting to think seriously about upgrading the program to a two phase process.

There are too many stop situations scattered among the Bans to continue the present system.

Phase 1 would collect Usernames from the Bans table, and Phase two would read the stored Usernames instead of re-generating the query, as at present.

In the two phase scenario, if there is a post in an ID, then that Username would be skipped and the program would fetch the next available Username from the previously collected store. This upgrade would allow the entire Bans table to be processed at one time, and the records left behind would be the ones with posts, which you could examine when you have time.
***

In an experiment, I discovered that the existing version of the program can collect all Usernames in the Bans table.

I'll still need to modify the program to read the Usernames from an external file.

Of the alternatives, changing the program to read an external file is definitely easier.

Even then, the program still needs to be modified to skip ID's that are identified as questionable, instead of halting.

Update at 17:36 local time ...

An experiment collecting Usernames from the Bans table using the existing program was a success.

I'll proceed to install updates to enable WBA to read Usernames from an input file. The first test is 150 User names.

Of those, some significant number will not qualify for recovery because they have posts.

The test run will confirm the ID's with posts will be left untouched.

I estimate the first test run to be in about a week, on the outside.

If Elon Musk and his team can land SN15 (let alone fly it ** again **) I should be able to upgrade WBA to read an input file!

(th)

Last edited by tahanson43206 (2021-05-07 15:41:16)

SpaceNut · 2021-05-07 19:48:58

Yesterday evening I helped move the subaru into the mechanics garage bay so that it could eventually have the engine put back into the car. Was not expecting the mechanic to work on it right away but to my chagrin the vehicle had its engine in and connected up to its transmission plus. It was still awaiting filling of the coolant, oil and placement of the battery back into it so as to be able to start it after having rebuilding it over the winter months.

Well it had been outside with water in the heater core from the last time it ran back in September or October of last year and was hopeful that it had survived but it did not. It did start right up smoked for a bit as it burned off the oils and transmission fluids that had ended up on it while taking it appart. Will need to bypass the heater lines until I get time to replace it but its summer so AC is a bit more important to refill for now.

It ran for about an hour idling without over heating even with the leaking heater core so will still need to test it some more before trying to make use of it for real but its looking good so far.

tahanson43206 · 2021-05-08 05:07:55

For SpaceNut re Subaru test run after overhaul! Congratulations (both to you and your mechanic) for your (apparently) successful enterprise!

This may not be something you can take time for right now, but if you ever get a few minutes free, I'd be interested in seeing your extrapolation of what an engine rebuild would be like on Mars. You would definitely need a heater to keep passenger cabin warm, but I don't foresee much need for air conditioning.

AC in the floating habitats on Venus might be needed, depending upon where the habitat is floating.

Please ** do ** update this (topic within a topic) when you are brave enough to take the Subaru out for a spin.

(th)

tahanson43206 · 2021-05-08 05:42:48

For SpaceNut re ID Recovery

It appears the existing WBA can collect all the Usernames in the Bans table in a single sweep using a script of five lines.

I'm planning to give the script a test later today.

I already have a 150 user name list created manually, as a test.

As reported earlier, changes to WBA to ** use ** the list will take a few days.

When they are working, it should be possible to sweep the entire Bans table in one operation, saving all ID's with posts, and recovering those without.

The number of Users with posts greater than 0 is down slightly (from 36 pages to 35) as you have (carefully) pruned spammers.

35*50 >> 1750 (max) That's a respectable number of ** real ** ID's. Some of those are banned, but I'll bet the non-bans will be 1,000 or more.

My immediate goal for the ID Recovery is to reach 5,000 .... we're standing at 4930 right now.

The goal I have in mind beyond 5,000 is 7,800 ... that is the number of plots I'd like to see filled with active contributors in My Hacienda.

Edit at 9:05 local time

Total Command Lines found: 12
Total input Lines in script: 47
Number of ID's processed: 73
Starting Number: 1
Last Number of Run: 73
Summary for Web Automation Report for 05-08-2021 at 09:03:29
Average time of Loop: 00:00:20
Average time of Loop: 00:00:20
Total time of Run: 00:28:38
Successful Run. No Warning Errors.

The run above collected all Usernames in the Bans table.

I'm thinking of setting up a run using the names as input to the next version of WBA.

Edit at 9:51 local time ...

3608 Usernames are collected ... if each one takes 3 minutes (based upon previous times) it would take 7+ days to process each one.

However, many of those will not qualify for recovery (because they have posts) so the actual time will be less

Still, that number of days gives a sense of the time scale involved.

(th)

Last edited by tahanson43206 (2021-05-08 07:55:00)

SpaceNut · 2021-05-08 16:52:53

Today the mechanic did bypass the heater core as he wanted to see if the rebuild was going to hold up and after 2 hours of running idle it still had not gotten hot with proper engine fans coming on as they should have. The real tell will be is when the engine cools does it need more coolant.

tahanson43206 · 2021-05-08 19:34:26

For SpaceNut re #1115

Thanks for the good news the renovated engine held up for a 2 hour test!

If I am following your report correctly, the radiator is running properly, but the heater has a leak?

I think that might be why you said you were more concerned with AC right now than the heater.

***
Upgrade of WBA is coming along ... the Username file is being loaded into an array for use at runtime.

I'll have to create a new script command to pull a line of data from the data array instead of asking FluxBB for a fresh query.

***
Interesting discussion about legs for Starship ....

(th)

tahanson43206 · 2021-05-09 12:23:47

For SpaceNut re ID Recovery ...

Best wishes on continued fine tuning for the Subaru ....

***
WBA Level 1.1 is ready for testing.

It is highly likely I overlooked something in adding this new functionality into a running program, so I'll start in Step Mode and fall back to Debug Mode if necessary.

Nice to see the link that Louis provided to Elon's SNL appearance ... I only watched a bit of it in favor of finding a faster computer for the video.

However, I am impressed by the man's self confidence as he stepped out on that stage.

(th)

tahanson43206 · 2021-05-09 20:43:20

For SpaceNut re ID Recovery ...

Best wishes for a productive work week ahead!

***
I found a problem with my upgrade that I'm not going to call a mistake ... I'm calling it a learning opportunity ... The language I use has the ability to allocate memory dynamically at run time. Apparently I've never had a need to use the feature before. I wanted to be able to hold the entire set of Usernames from a query in memory at run time, and since that is a variable number, I thought I'd just extend the memory when I need it.

What I didn't know (having never used the feature before) is that the contents of a previously allocated section of memory are discarded when the new allocation is made. That makes perfect sense (in hindsight) but the solution is to create a primary memory and a backup one, and copy data from one to the other as the extension is needed.

A revised version of WBA is ready for testing tomorrow. I'm thinking of running a test with 149 Usernames. That should run for about 8 hours.

My guess is that about half of the test ID's can be upgraded, but we'll see.

Edit Next day after test ... Now a ** real ** error showed up ... Back to the debugger ... the symptom is a status message indicating the wrong data source.

(th)

Last edited by tahanson43206 (2021-05-10 06:36:09)

tahanson43206 · 2021-05-10 09:20:50

For SpaceNut re ID recovery ....

This post is reserved for updates on resolution of the latest problem with WBA Level 1.1, and any test results that may become available.

i'm ** definitely ** interested in finding out why Null was delivered to the RS232 program for output to the browser at run time, in the latest test.

The Data Array is now filling up correctly, but apparently(obviously) the fetch isn't.

***
Just FYI ... no traffic at NewMarsMember * gmail.com this morning. Noah must be studying like crazy again! All Good!

***
WBA Level 1.1 passed non-live testing. Something may still happen in the live interaction with a ** real ** web site.

Edit at 15:22 local time .... the live test passed the first ID ... 4931 is upgraded and ready for your review

How the program performs when exceptions occur remains to be determined ... the current run is scheduled to try for 5000.

***
Edit at 15:45 local time ... WBA 1.1 is failing to make the transition from the first ID in a series to the second.

There's a problem with moving the pointer to select the next Username.

4931 and 4932 were completed and they are ready for your review.

Will try again tomorrow. Upon reflection, I suspect I did not create a line of code to increment the pointer.

With everything else going on, including problems with memory allocation, I might easily have overlooked that small but essential "feature".

(th)

Last edited by tahanson43206 (2021-05-10 13:57:49)

tahanson43206 · 2021-05-10 14:03:24

This is a new subtopic that I am opening in Housekeeping because I don't want to clutter other topics with discussion ...

I'm interested in the opinion of Calliban (first and foremost), kbd512, GW Johnson and Oldfart1939, and anyone else who supports the proposition:

Nuclear Power is Essential for Life Away from Earth

I would propose a companion: Companion to Nuclear Power is Essential for Life Away from Earth

SpaceNut is a key player in this initiative ... I'd like to suggest we limit participation to the four persons named above for the primary topic.

Everyone would be encouraged to participate in the Companion.

I'm looking for hard facts to develop in the primary.

I'd like to see the primary topic develop with hooks for finding specific items easily.

The forum software is lean and there aren't a lot of bells and whistles, but we humans have already demonstrated the ability to work with the environment to make it more useful for the longer term, as well as for the immediate discussion that might be in progress.

Edit#1: The proposition I hope to develop is that with sufficient power available, a human can live comfortably away from Earth, ** without ** resupply from Earth. The early ideas/explorations of trying to live in a space habitat without resupply envisioned all sorts of low energy solutions.

I'd like to propose design of a living space away from Earth that is NOT limited in power, and therefore can perform such operations as completely disassociating atoms from others in molecules, so that ** all ** waste from the living space is reduced to the pure form that can then be fed back into the environment as needed.

(th)

Last edited by tahanson43206 (2021-05-10 14:08:06)

SpaceNut · 2021-05-10 18:39:56

Not sure that I can add all that much to the topic but louis will always so no to nuclear....

I have unbanned the table all but for the 3 duplicates with different emails and a common IP address shared by them 95.65.79.205

The 3 that show up in the user tables are still banned as well...

tahanson43206 · 2021-05-10 20:22:59

For SpaceNut re ID recovery ...

Thanks for taking care of most of the new upgrades!!!

I found that duplicates were causing a halt, so I investigated and discovered that spreadsheet software can detect them. The capability is available in all recent brands of spreadsheet. I used Libre and removed several hundred duplicates from the list of Usernames collected a few days ago. The run should now proceed at this end, but the duplicates may still be present in the Bans table... the original count of usernames was 3608 and the new coujt is 3430, so you can expect to see just under 200 duplicates in the days ahead. I'll try to run 100 ID's per day.

Halts can still occur, but they should be less frequent.

***
Louis is a valuable member of the forum! I've noted before that he is particularly good at inspiring our knowledgeable authors to deliver their best work. Without Louis, I suspect this forum would not see the variety of posts of the quality so often achieved.

It is possible no one (other than you) actually reads Housekeeping, so my proposal may flow under the bridge like everything else << grim >>

(th)

tahanson43206 · 2021-05-11 06:58:47

For SpaceNut re ID recovery ...

Last night's run was set for a goal of 5,000. 4983 was the high water mark achieved. An Internet delay appears to have been the cause of a halt.
I restarted the run to try for 5,000, and the Username at the time of the halt proceeded normally.

If we schedule 100 ID's per day, as was the norm with Not Verified, it will take (on the order of) 36 days to work through the Bans, and that's assuming you don't add more, which is certainly possible.

Edit 9:30 local time ... the Milestone of 5,000 recovered ID's was reached after a restart this morning.

I'll attempt a run of 100 ID's starting this evening. The Bans table should be visited completely by (US) Flag Day.

Total Command Lines found: 108
Total input Lines in script: 535
Number of ID's processed: 17
Starting Number: 4984
Last Number of Run: 5000
Summary for Web Automation Report for 05-11-2021 at 09:35:31
Average time of Loop: 00:02:54
Total time of Run: 00:59:51
Successful Run. No Warning Errors.

(th)

Last edited by tahanson43206 (2021-05-11 07:41:30)

tahanson43206 · 2021-05-11 17:45:00

For SpaceNut ... tonight's run is under way ... 5001-5100 is the setting

The very first ID had a date separation of 3 years+, so I set the gap at 1300 to cover most cases.

What I'm looking for is how the program handles an ID with posts ... what it is ** supposed ** to do is report the exception, reset the pointer to the data array (where the Usernames are stored) and try the next Username. I hope that operation goes as planned, but failure would involve a halt, so I'm prepared for that to happen. Otherwise, the run should be for 300 minutes or 5 hours or so.

***
For SpaceNut ... the ID that had the long gap between Registration and Last Visit had an email domain of "top"

It turns out that is a "real" Internet Domain:

.top
From Wikipedia, the free encyclopedia
Jump to navigationJump to search
.top
.top domain.svg
Introduced November 18, 2014
(General Availability)
TLD type Generic top-level domain
(gTLD)
Status Active
Registry .top registry
Registry website www.nic.top
.top is a generic top-level domain, officially delegated in ICANN's New gTLD Program on August 4, 2014. The extension is managed and operated by the .top registry,[1] and can be registered by anyone since November 18, 2014.
.top is blocked by default by Snort rules.[2]

We are left with a mystery as to why that user returned to log in after three years, and did not post anything.

(th)

Last edited by tahanson43206 (2021-05-11 17:52:56)

SpaceNut · 2021-05-11 19:37:07

Did the unban of 4955 through to 5001 but then a duplicate popped up so stopped doing the unban for that neat duplicate that had no IP so did 5002 through to 5012... and saw your run was still completing
and the next duplicate has occurred at 5040.....

New Mars Forums

Announcement

#1101 2021-05-05 17:48:49

Re: Housekeeping

#1102 2021-05-05 17:55:06

Re: Housekeeping

#1103 2021-05-05 18:01:14

Re: Housekeeping

#1104 2021-05-05 18:13:39

Re: Housekeeping

#1105 2021-05-06 06:20:47

Re: Housekeeping

#1106 2021-05-06 17:35:02

Re: Housekeeping

#1107 2021-05-06 17:44:15

Re: Housekeeping

#1108 2021-05-06 21:27:18

Re: Housekeeping

#1109 2021-05-06 23:39:35

Re: Housekeeping

#1110 2021-05-07 05:25:35

Re: Housekeeping

#1111 2021-05-07 06:06:30

Re: Housekeeping

#1112 2021-05-07 19:48:58

Re: Housekeeping

#1113 2021-05-08 05:07:55

Re: Housekeeping

#1114 2021-05-08 05:42:48

Re: Housekeeping

#1115 2021-05-08 16:52:53

Re: Housekeeping

#1116 2021-05-08 19:34:26

Re: Housekeeping

#1117 2021-05-09 12:23:47

Re: Housekeeping

#1118 2021-05-09 20:43:20

Re: Housekeeping

#1119 2021-05-10 09:20:50

Re: Housekeeping

#1120 2021-05-10 14:03:24

Re: Housekeeping

#1121 2021-05-10 18:39:56

Re: Housekeeping

#1122 2021-05-10 20:22:59

Re: Housekeeping

#1123 2021-05-11 06:58:47

Re: Housekeeping

#1124 2021-05-11 17:45:00

Re: Housekeeping

#1125 2021-05-11 19:37:07

Re: Housekeeping

Board footer