Housekeeping

tahanson43206 · 2021-08-04 07:36:36

For SpaceNut re ID recovery

This post is reserved for updates on a set of runs to update 117 ID's ...

The first step will be verify, to be sure the names collected with Zero posts are still in the users table.

Total Command Lines found: 24
Total input Lines in script: 80
Total input Lines in data: 3870

Number of ID's processed: 120
Starting Number: 1
Last Number of Run: 120
Summary for Web Automation Report for 08-04-2021 at 10:32:03
Average time of Loop from Main form: 00:00:24
Total time of Processing: 00:48:33
Total time Program was Active: 00:50:59
EOJ Msg#1: Successful Run. No Warning Errors.

The output of the verify run is a list of candidates who are NOT in the users table.

In this case, there were Zero usernames NOT in the users table.

The next run will be over the same set of usernames, but this time we'll be confirming they are in the Bans table.

Update at 12:35 local time ... SpaceNut, this is really inside baseball ... for the record only ...

I just saw a failure mode during the Bans run that is totally new (to me for sure).... The mouse positioning algorithm is working reliably, with failures only every 10,000 tries (or so). However, in **this** case the mouse position was accurate, but the browser software failed to clear the target field before the mouse arrived. The failure was visible as the insertion of a user name into one left over from the previous loop.

If this situation were to occur more frequently, it would be possible to add delay ahead of the click, or perhaps to add a field clear operation at the site of the click. In any case, the safety checks caught the error and halted the run.

Update at 12:50 local time

After restart, and covering the same 120 usernames, we have an All Clear!

Total Input Lines 654
Total Output Lines 137
Lines containing String One 120
Lines containing String Two
Lines containing String Three
Successful Run. No Warning Errors.

117 usernames will be fed into tonight's run to 13,500.

(th)

Last edited by tahanson43206 (2021-08-04 10:51:55)

tahanson43206 · 2021-08-04 13:12:36

For SpaceNut ....

This post is totally about "housekeeping"

https://www.yahoo.com/news/decades-wood … 45837.html

The story at the link above is about a ? neighbor ? who's been living on someone else's land for decades.

There are some who want him to live out his days where he is, but it appears that the township is fed up.

The gent is cooling his heels in jail at the moment.

A solution might be for a consortium to buy the land from the current property owner, and set up a suitable "camping" facility with sewerage (septic tank) and whatever else is needed to meet code.

That would be the best possible outcome, but it would require generosity, and ** that ** is in short supply on Earth these days.

(th)

kbd512 · 2021-08-04 14:24:50

tahanson43206,

If SpaceNut shows me what he's doing, then I will record that meeting, use it to generate a script for a browser-based automation tool to execute, and then we can dispense with the manual labor associated with doing that.

tahanson43206 · 2021-08-04 16:00:34

For SpaceNut re kbd512 offer in Post #1428

Please consider taking kbd512 up on his offer. We have 3,300 ID's to upgrade, and I'm expecting to deliver 3,900 more in the weeks ahead.

I (obviously) have no idea what kbd512 is talking about, but whatever it is, it sounds terrific!

(th)

tahanson43206 · 2021-08-05 08:13:50

For SpaceNut re ID Recovery ...

WBA completed a run to 13500 last night, with only one hiccup. I'll start verify runs for the next batch of 100 shortly.

it is good to be back engaged in the project. Thanks for keeping the intake hopper full!

kbd512 has offered to take the finishing process off your hands. Please write up what is needed to get the procedure to kbd512.

It sounds as though he is willing to look at it to see if he can automate the procedure. That would be a nice advance that could be shared with FluxBB.org when and if he gets it working.

Update at 12:02 local time ...

Verify in Users Table completed with no missing items.

On to the Bans table verification. This is the run that is more interesting (to me for sure!) ... This run reveals decisions SpaceNut made in his painstaking manual review of Usernames to see if they are spammers or potentially real people. The spammers have been (and still are) very creative is simulating real people, so SpaceNut's task is not easy. On the other hand some spammers seem to have been sloppy in their automation of generating fake names and email accounts, so they are relatively easy to identify.

The work SpaceNut is doing is most definitely under appreciated, so this acknowledgement will have to be his reward.

Update at 12:59 local time ...

The Verify Bans run was successful. All Usernames were in the Bans table.

The verified input will be used for tonight's update.

(th)

Last edited by tahanson43206 (2021-08-05 11:00:07)

tahanson43206 · 2021-08-06 05:53:18

For SpaceNut re ID Recovery

WBA made it safely to 13600 last night. The Internet seas were glassy smooth. The Moon is in waning crescent.

Today's verification runs will start shortly.

Update at 13:12 local time

The verify run over 110 usernames found them all in the Users table. One down and two to go.

Bans verify will start shortly.

Update at 16:55 local time ... The Bans table verification completed.

All 110 Usernames tested were found in the Bans Table.

All clear for tonight's update run.

***
kbd512 offered to help with finishing the TestID's to move them into full Member status.

Are the two of you communicating privately? Do you have questions about what kbd512 would need?

(th)

Last edited by tahanson43206 (2021-08-06 14:58:36)

SpaceNut · 2021-08-06 16:39:09

Patriothmw, dropped by and appears to be still logged in but has not posted..

tahanson43206 · 2021-08-06 17:33:19

For SpaceNut re #1432

The P section is still a ways out. As long as you keep feeding WBA with juicy Spammer ID's we'll get there!

Today's batch of 100 verified usernames are queued up for tonight's run!

***
Looking for your thoughts about kbd512's offer. You very well may prefer not to delegate that responsibility, but I am hoping you will at least acknowledge kbd512's offer. he is your backup as Administrator, so preparing him to take on your role in your absence is part of your job description.

(th)

SpaceNut · 2021-08-06 19:01:54

Its not all that hard as its under the administration, side bar ban then fill in the test* to search to produce the ban table of names with any thing containing test …. and at the outputted table that appears you have the choice to edit or remove ban. Select the remove and wait for the refresh to execute the function and then you would do a double back arrow selection to get back to the table to select the next name from the list...after you think you have done all of those that you want then verify the names are in the user table. If you find any that are duplicate names but the email has not change use the edit function to change the name of the user and then the email to be able to clear it later from the banned table.

tahanson43206 · 2021-08-06 19:11:41

For kbd512 ... It is possible you do not read the Housekeeping topic, but if you do, it ** looks ** to me as though SpaceNut has described the procedure for upgrading TestID's in the Bans table to full member status.

As you can see, there appears to be a bit of decision making involved, since the version of FluxBB that's been in use for many years allowed duplicate entries in the Bans table.

Never-the-less, if you can work out a way to automate the part of the procedure that does NOT require decision making, then that would increase the member candidate pool from the current 10,000 to over 13,500. I am adding 100 more TestID's to the Bans table every day, as long as SpaceNut keeps feeding ripe, juicy spammer ID's to the maw of the grinding process.

For SpaceNut ... the run for the next 100 will start now...

update at 23:55 local time .... the TestID's waiting for kbd512 to upgrade them from Banned to Members stands at 13700.

Completed Sequence for ID: 13700

Total Command Lines found: 112
Total input Lines in script: 341
Total input Lines in data: 3653
Number of ID's processed: 100
Starting Number: 13601
Last Number of Run: 13700
Summary for Web Automation Report for 08-07-2021 at 00:37:29
Average time of Loop from Main form: 00:01:29
6 Date Exceptions were recorded.
Total time of Processing: 02:29:20
Total time Program was Active: 03:24:09

(th)

Last edited by tahanson43206 (2021-08-06 22:41:41)

tahanson43206 · 2021-08-07 08:29:52

for SpaceNut re ID Recovery

This post is intended to cover the run from 13701-13800

Step 1 is Verify Usernames with Zero posts are still in the users table. Starting now ...

(th)

SpaceNut · 2021-08-07 09:18:13

Have been busy with the 12 pages of Letter L moving them into the banned. Accounts have been zeroed for the spammers and real accounts are locked.
Found what appears to be an account that I bumped but did not lock fully which started a topic which you reposted content to and from there website.

tahanson43206 · 2021-08-07 09:47:27

For SpaceNut re #1437

Thanks for preparing input for WBA to work with! The Verify run #1 just finished. All candidates are still in Users.

I'll start the Bans Verify next.

I vaguely remember that encounter << grin >>

Hoping your weekend is productive and enjoyable.

Your visit with your brother and relatives was a recent high spot in recent times. I hope he continues to recover, albeit slowly, and that everyone else protects themselves from the latest threat. Our area has gone back to voluntary mask wearing. Businesses here seem to be willing to ask for masking. I've taken to carrying my mask in a pocket so it is handy depending upon the situation. Generally, I figure if an employee is wearing a mask, it is a courtesy to wear one also.

Update at 12:41 local time ... The Bans run showed that all candidates are in the Bans table.

They will be queued up for tonight's dash to 13800.

(th)

Last edited by tahanson43206 (2021-08-07 10:42:45)

tahanson43206 · 2021-08-07 22:26:50

For SpaceNut ...

This message showed up today.... WBA had completed it's run and the FluxBB system logged me off automatically.

When I clicked on the screen to log back in, this message appeared:

Update FluxBB
Your FluxBB database is out-of-date and must be upgraded in order to continue. If you are the board administrator, please follow the instructions below to complete the upgrade.
Note: This process is for board administrators only. If you are a member there is nothing to worry about - the forums will be back shortly!
Update FluxBB
This step is for the board administrator only!

I have not seen a message like that before.

***

Completed Sequence for ID: 13800
Total Command Lines found: 112
Total input Lines in script: 341
Total input Lines in data: 3553

Number of ID's processed: 100
Starting Number: 13701
Last Number of Run: 13800
Summary for Web Automation Report for 08-08-2021 at 00:15:18
Average time of Loop from Main form: 00:01:29
9 Date Exceptions were recorded.
Total time of Processing: 02:29:25
Total time Program was Active: 03:08:22

The Internet was smooth as glass this evening! It's a new Moon. Skies were clear.

(th)

tahanson43206 · 2021-08-08 07:14:15

For SpaceNut re ID Recovery

First ... it was nice to see Mr. Burk has updated the FluxBB package! I hope one of us has time to find out what improvements were included.

Second ... kbd512 has kindly offered to help finish the TestID's that are Banned and need to be upgraded to Member status.
You have provided instructions on what needs to be done. However, it would probably help if you were to touch base with kbd512 from time to time.
He could start with conversion of one TestID to member status, to show he understands the manual process. That would be a good sign that your instructions were helpful.

Finally ... this post is reserved for updates on tonight's run to 13900. The steps will include Verify Users and Verify Bans, followed by Upgrade.

Update at 10:24 Local time: The verify users run was successful. All candidates are still in the Users table.

Starting verify Bans now.

Update at 12:01 local time ... The Verify Bans run ended with all present and accounted for.

The candidates are queued up for tonight's dash to 13900.

(th)

Last edited by tahanson43206 (2021-08-08 10:02:25)

SpaceNut · 2021-08-08 07:47:14

Scanned ahead for duplicates and here is the first pair

TestID10846 testid10846@test.com Never Upd: 2021/07/06 SpaceNut Edit | Remove
TestID10846 daniel.bukus@vinbel.com Never SpaceNut Edit | Remove
TestID10847 testid10847@test.com Never Upd: 2021/07/06 SpaceNut Edit | Remove
TestID10847 tfwvsg@hagemeyer14.silkbeachtowels.com Never SpaceNut Edit | Remove

To remove the duplicate edit the one with the original email which shows second
change the user name to test* A so that it will think its a new name and then edit the email in the same manner as the one which was coverted

username ex. TestID10846a
email EX. testid10846A@test.com

Hit the submit to save the changes the table will when you refresh the table its going to show the new id. in the banned listing

Then you can do the remove the ban for the username of the first followed by the second and its flushed out of the system as the second was not in the creation table.

Just caught Serviceabj peeking … spammers are catching onto the accounts disappearing slowly...

tahanson43206 · 2021-08-08 20:14:46

For SpaceNut re Serviceabj ....

That window may be closing slowly compared to the size of the problem, but it is closing in 100 step chunks, and will reach the goal in a few weeks.

At that point (thinking ahead) I'm planning a re-run of the script to find users with zero posts. The only ones it finds ** should ** be ones you've decided to keep. After that we can begin to put the squeeze on ID's with ghost posts.

***
Here's something to put in the back of your mind in case something comes of it. The speaker at Saturday's online meeting of the North Houston chapter of the National Space Society is/was an impressive young woman who's been thinking about a career relating to space for a while. She seems to be leaning toward space medicine. However, that by itself is not what I think is worth mentioning. The young woman is attempting to encourage elementary aged students to think about space. Others have done similar things, over the years. The impulse may last for a while but then give way as the demands of college and career grow. it crossed my mind that My Hacienda might (somehow) fit into an education campaign that reaches young people. After all, ** they ** are the ones who will be going, either as job creators or as job holders.

The YouTube video should be posted by now: www.northhoustonspace.org

(th)

SpaceNut · 2021-08-08 20:41:07

Just processed half of the M's resetting the ghost post spammers and locking accounts of posters that have not dropped by for some time now.

tahanson43206 · 2021-08-08 21:14:42

Thanks for getting a head start for the week ahead! Reaching all the way to the M's means we must have next week covered.

Nice to see you're taking a look at our little goblins as well << grin >>

The current active members seem to be in good form in recent times. I'm glad to see GW's wife is progressing, and that kbd512's wife's doctor has good news as well. I'm hoping OF1939 is continuing to recover from his bout with Covid and hospital stay. That would knock the wind out of anyone, and ** he ** is in his 80's ... I hope that he recovers fully and is able to resume work on the expedition design.

We haven't heard from Elderflower in a while ... he was getting "up there" as well ...
Last post was 2020-10-31 05:41:02

Mars_B4_Moon has ** really ** been adding to the flow in recent times. I get a kick out of seeing some of the old topics he brings back to view.

He sure does seem to have in in for Reddit << grin >>

Update at 00:12 local time ... WBA just finished so I'll post the summary before closing the laptop for tonight.

Completed Sequence for ID: 13900
Total Command Lines found: 112
Total input Lines in script: 341
Total input Lines in data: 3453
Number of ID's processed: 100
Starting Number: 13801
Last Number of Run: 13900
Summary for Web Automation Report for 08-09-2021 at 00:13:15
Average time of Loop from Main form: 00:01:29
8 Date Exceptions were recorded.
Total time of Processing: 02:29:42
Total time Program was Active: 02:38:07

The Internet was glassy smooth again this evening. The Moon is New.

(th)

Last edited by tahanson43206 (2021-08-08 22:16:56)

tahanson43206 · 2021-08-08 22:17:55

This post is reserved for a run to 14,000 planned for Monday evening.

Verify runs will be performed after sun up ...

The first verify run produced a mystery ... it flagged KAYLindsay as not in the users table, but it is there.

There is a lot that goes on to verify a username. Every bit has to travel from the laptop through the Internet to the server and back, and it then gets passed around in the laptop multiple times. It is amazing so many bits get transferred without error every minute of every day, so that we Internet users take reliability for granted. But the Internet itself was constructed at a time when reliability of hardware was NOT great, or even good, so there are checks built into the traffic flow.

I'll ignore the error reported by Verify users, and see what happens with Verify Bans.

update at 11:54 local time ... The Verify Bans Table process completed without incident.

The 100 verified candidates will be fed into tonight's run to 14,000.

(th)

Last edited by tahanson43206 (2021-08-09 09:56:08)

kbd512 · 2021-08-08 22:51:01

tahanson43206,

Apologies, but I'm just now reading SpaceNut's instructions. I kept checking my E-mail for instructions about what SpaceNut wanted me to do, but received none, so I figured he wanted no action taken. The post in here was not addressed to me, but I found it after you commented on it.

tahanson43206 · 2021-08-09 07:21:03

For kbd512 re #1446

Thanks for your follow up !!! It may take a while for you to decide on how you want to proceed, but it sure will be good to see the count of finished ID's start climbing up from 10,000! There are three stages to the process ... SpaceNut performs the first operation, which is to select candidates by banning them. This operation cannot be automated, because SpaceNut uses his experience and judgement to decide who is a spammer and who is not. I've been handling the middle phase, which yields TestID's that are in the Bans table. The final product is a TestID ready for use by a real person. We have added three new members since this process started, and have another in the queue.

The dilemma we face is that the members we've admitted so far are all in various stages of college (PhD candidate to freshman), and they are ALL super focused on their studies. Meanwhile, our existing members are keeping the flow of discussions going, in addition to project work by RobertDyck and OF1939.

(th)

kbd512 · 2021-08-09 18:54:16

tahanson43206,

I have Katalon (Selenium). I'm going to use the tool to create a browser automation script. This is a very common practice in website and web app testing, believe it or not. I need a quick demo of what SpaceNut wants me to do, and I also need to know about any exceptions to the sequence of actions he's taking. Can we set up a quick GoTo-Meeting or Zoom or whatever the cool kids are using these days? If not, can SpaceNut E-mail a file with a recording of the steps he's taking?

SpaceNut · 2021-08-09 19:13:51

Sorry for not sending an email as it slipped my mind....

Here is a step by step

Administration
http://newmars.com/forums/admin_index.php

Moderator menu
http://newmars.com/forums/admin_bans.php

Ban search
Enter search criteria
Username

enter Testid*
Submit search

This brings up the table of names banned that have been converted where the * is the number generated.

here is what you will see for the first entry in the table

TestID10001 testid10001@test.com Never Upd: 2021/06/26 SpaceNut Edit | Remove

select the Remove link at the end

At that point the del ban function executes and does a screen refresh to the Ban search entry form

To go back 2 screens to get the table once more for the next name in the list or enter the enter Testid* once more in the search field to get the table refreshed.

rinse repeat until all names are removed from the ban table.

Verify the username is in the user list and listed as member and should not still have a ban status if all works
ex.
TestID10000 Member 0 2017-04-26

kbd512 · 2021-08-09 19:26:19

SpaceNut,

Okay, I got it. One last question, though:

If I automate this, it's going to go through all of the accounts really fast. Will the server stop responding to the requests?

Also, can you give me a specific range of Test IDs to execute this script for?

It looks like I'm simply iterating a counter and going through a sequential list of IDs. Is that correct?

New Mars Forums

Announcement

#1426 2021-08-04 07:36:36

Re: Housekeeping

#1427 2021-08-04 13:12:36

Re: Housekeeping

#1428 2021-08-04 14:24:50

Re: Housekeeping

#1429 2021-08-04 16:00:34

Re: Housekeeping

#1430 2021-08-05 08:13:50

Re: Housekeeping

#1431 2021-08-06 05:53:18

Re: Housekeeping

#1432 2021-08-06 16:39:09

Re: Housekeeping

#1433 2021-08-06 17:33:19

Re: Housekeeping

#1434 2021-08-06 19:01:54

Re: Housekeeping

#1435 2021-08-06 19:11:41

Re: Housekeeping

#1436 2021-08-07 08:29:52

Re: Housekeeping

#1437 2021-08-07 09:18:13

Re: Housekeeping

#1438 2021-08-07 09:47:27

Re: Housekeeping

#1439 2021-08-07 22:26:50

Re: Housekeeping

#1440 2021-08-08 07:14:15

Re: Housekeeping

#1441 2021-08-08 07:47:14

Re: Housekeeping

#1442 2021-08-08 20:14:46

Re: Housekeeping

#1443 2021-08-08 20:41:07

Re: Housekeeping

#1444 2021-08-08 21:14:42

Re: Housekeeping

#1445 2021-08-08 22:17:55

Re: Housekeeping

#1446 2021-08-08 22:51:01

Re: Housekeeping

#1447 2021-08-09 07:21:03

Re: Housekeeping

#1448 2021-08-09 18:54:16

Re: Housekeeping

#1449 2021-08-09 19:13:51

Re: Housekeeping

#1450 2021-08-09 19:26:19

Re: Housekeeping

Board footer