Housekeeping

SpaceNut · 2021-01-18 18:36:44

These blank user name fields are not user accounts but email address blocking for registrations meant to stop would be spammers that would use them.....By leaving these and adding more of the ones we are finding would be a continued method to even alert us of those trying to use them for our new process.

Just continue on the Not Verified accounts...

edit
did another 10 to create 2011 -2020

tahanson43206 · 2021-01-18 18:49:39

For SpaceNut re #701 ...

Interesting! Thanks for the clarification!

Question ... now that we are manually processing applications, and will be hand typing email addresses into the new user accounts, do we still need the blockers?

Edit#1: Answer: After thinking about the situation for a while, I realized that the blocks need to remain.

The reason is that anyone can change email address after securing membership. Under ordinary circumstances that would not be a problem, since folks ** do ** change email providers from time to time, and need to be able to set up a new email address.

However, the blocks in place will insure that specific email addresses are not used.

Thanks again for the clarification.

(th)

Last edited by tahanson43206 (2021-01-18 19:04:39)

tahanson43206 · 2021-01-19 09:43:27

For SpaceNut ... Visman at FluxBB.org gave us a ** very ** nice set of screen shots showing how to ban fifty accounts at a time.

In a moment I'll be back with a link to the post.

In the meantime, here is a copy of my reply:

For Visman re #4 ... Thanks very much!
50 at a time is a ** big ** improvement!
I'll pass the link to your reply immediately.
FYI .. the Admin ran the query you gave us yesterday, and several previously unknown ID's showed up. It turned out those were created by another Admin, log ago and far away, to try to prevent use of certain email addresses.

Here is a link to Visman's reply: https://fluxbb.org/forums/viewtopic.php … 194#p66194

Edit#1: The example provided by Visman shows a screen that may be available only to Admins. I hope it is available to you.

As a suggestion ... it might be worth seeing if you can ban 50 qualifying spam ID's in a single execution of the screen.

At this point, renaming the ID's can be left for later.

If you can ban 50 at a time, and if it takes five minutes to set up each set, then conceivably the entire set of qualifying ID's could be processed in a few weeks, instead of the three years we are facing today.

I would still prefer a single command to ban all of the qualifying ID's in a single operation, but this solution from Visman is a 50:1 improvement so we could probably reach the immediate objective with that.

Edit#2: ID's are ready for unbanning: 2011-2020 and 1431-1452

(th)

Last edited by tahanson43206 (2021-01-19 15:35:56)

SpaceNut · 2021-01-19 18:44:45

The screen you can see if its go posts but yes its the same one for changing the membership status from not verified to newly registered, member and moderator.

The normal user list does show the ban condition with post and registry date but thats all that can be done with that screen.

Looking for 1431at the moment but got all of the others done

edit
found it as 0431 in the table.. all fixed now...

tahanson43206 · 2021-01-19 19:57:31

For SpaceNut re #704

Thanks for catching that pesky 0431 ...

The Count is up over 16,740 now ... slow but steady progress.

Apparently the Administrators see screens not shown to anyone else.

I do not see a screen like the one that Visman showed.

He spoke of doing 50 bans at a time.

(th)

Last edited by tahanson43206 (2021-01-19 19:59:59)

SpaceNut · 2021-01-19 20:19:51

The 50 ban at a time comes from the settings for post display on a page. The names of users are treated the same as a post.

Finished going through the alphabetical names starting with A out to page 20 with banning's.

Thanks for the help...

I wonder how KBD512 is doing on a script to correct the shifting extra tag portion of commands for the BBcode conversion caused issue as there is a bit of work there for sure still if we must do it manually....

tahanson43206 · 2021-01-19 21:44:07

For SpaceNut ... your mention of kbd512 brings to mind this evening's meeting of the Local Linux User Group. The leader is a (now retired) professor of computer language instruction at a local community college. One of the attendees is a gent who just retired after a career writing software going back to earlier than PL/1 (few remember that language), and continuing through years of C++, which I'm sure you know is at or near the top of technical difficulty. Both gents had experience with php, so I showed them one of the forum php scripts, and they explained nuances of the coding that are probably a short course in the subject.

I came away properly daunted by the difficulty of creating a php script capable of doing what is needed.

Each of the gents thought that manually plodding through all 19,000 ID's for three years is not the optimum way to go, and that a computer programmed solution would complete the task in seconds. However, each carefully excused themselves from further involvement, and closed with the observation that any such code would need to be tested thoroughly on a test system using a copy of real data to avoid damage to the production data.

I would guess that if kbd512 has thought about the problem, he might well have arrived at the same conclusion.

Your request for assistance reformatting the damaged posts is similar. Any code introduced into the production system needs to be thoroughly tested, and we do not currently have a test system.

If you ever get a chance to talk to Mr. Burk about problems of this nature, you might point out the need for a test system where any code along these lines can be thoroughly tested before it is let loose in the production system.

In the mean time, I have mentally prepared to maintain the current pace of 22 ID's per day (plus whatever you add from the pool of dangerous spam ID's). My immediate target is to try to finish the Not Verified ID's by June.

(th)

tahanson43206 · 2021-01-20 08:15:29

For SpaceNut re suggestion of Visman to use the Admin tool which shows 50 ID's at a time.

I took a look at that tool last night.

By any chance, did you notice the button at the bottom? That button allows you to ban all of the items collected on a page.

I noticed that in the selection pulled up by the tool in my test last night, there were two categories of ID I'm not wanting to ban.

The first category is the Test DI's ... those could be selected out using the alphabetic search window.

The second category is the Not Verified accounts ... I'll finish those by June, so would prefer we not ban them unless it is deemed necessary.

My assumption is that the Not Verified accounts are Not Verified because the test email to the submitted email address either did not arrive at all, or was not received. In either case, it seems to me the Not Verified accounts are not a risk.

The accounts that ARE at risk are the ones our spammer friends created AND verified, so they are capable of login at any time.

It may be possible to experiment with the parameters of the tool to show ONLY the accounts we want to ban in a page of 50.

At ** that ** point, one button click should be able to ban them all.

I got 353 pages in my test yesterday evening. That would be 17,600 or so. Of those, about 3,000 are Not Verified. About 1,500 are Test ID's.

The remainder of (about) 13,150 "may" be the ID's we want to ban to prevent unwanted logins.

Would you like for me to investigate further?

If we can find a way to collect just the ID's of interest, then we can ban 50 of them at a time, which would be 50 times faster than our current process.

(th)

tahanson43206 · 2021-01-20 08:16:19

This post is reserved for today's batch of Not Verified ID's: 1453-1474

***
For SpaceNut ... you've been busy! When I opened my Not Verified Query this morning, a group of ID's starting with "b" showed up ...

The procedure for dealing with those is slightly different from the procedure for Not Verified, but by now either one takes about the same amount of time.

!!! The bans table increased from 58 pages to 73 pages !!! That means you banned 73-58*50 ID's ... that's amazing!

I couldn't stand the mystery so ran the computation ... that's 750 ID's which would include the ones I found in the Not Verified Query.

If you can sustain that pace, you may have cleared the open spam accounts in a few weeks instead of three Earth years!

Just FYI ... I reported the progress to Visman.

***
For SpaceNut ... Here is an unexpected benefit of the new procedure, in which I am working on Not Verified ID's you've banned ...

The display of completed ID's holds steady, instead of scrolling. After thinking about that for a moment, I realized that I am pulling the ID from the front of the table and delivering it to the page where the results appear. The number of ID's in the query does not change.

Anything to make the job easier << grin >>
***
The first 11 of today's batch are in the bans table. It's time for a well earned break after 1.3 hours of wall clock time.
***
The last 11 went fast ... today's batch are ready for unbanning 1453-1474
***
Good luck with tonight's mass banning operation!

750 is probably exhausting but even a couple of hundred bans would be good to see!

(th)

Last edited by tahanson43206 (2021-01-20 15:08:54)

SpaceNut · 2021-01-20 20:49:11

1453 - 1474 are unbanned and moved to user list also converted 2021 - 2040 from banned list

tahanson43206 · 2021-01-20 21:16:03

For SpaceNut re ID recovery .... Thanks for 1453-1474

It's good to know the 2000 series is continuing to advance.

I'll be up there in a few weeks, right around the time Perseverance attempts a soft landing. A parachute would be a good idea for that mission << grin >>.

(th)

SpaceNut · 2021-01-20 21:23:58

Its also using the sky crane hover retro propulsion to allow a drop cable to lower it safely to the surface...

tahanson43206 · 2021-01-21 10:33:15

For SpaceNut .... today's batch of 1475-1496 are in progress

1475-1486 are completed in 1.1 hours ... time for a break.
***
The size of the bans table dropped from 73 pages to 72.
The size of the Not Verified query dropped from 58 pages to 57
The number of Registered users is increased to 16,768

*** Today's set of ID's reading for unbanning.

Just FYI ... Today's batch included the remainder of the "b" set you banned a few days ago. The Query is now serving Not Verified again.

(th)

Last edited by tahanson43206 (2021-01-21 11:16:58)

SpaceNut · 2021-01-21 19:01:04

finished with the unban and moving to the user list on 1475 - 1496

edit

converted from the A's id 2041 - 2050 did unban and moved back into the user list....

tahanson43206 · 2021-01-21 19:50:51

For SpaceNut re #714 ...

Thanks for taking care of the latest ID's.

The total of Registered ID's is continuing to increase: Total number of registered users: 16,768

We started at 16,494. We'll keep adding ID's to the total as long as there are available ID's in the Not Verified collection.

(th)

tahanson43206 · 2021-01-22 06:52:48

This post is reserved for today's batch of Not Verified ID's, from 1498-1518

The near term goal of reaching 2000 is in sight but some weeks away.

*** 11 ID's were processed ... time for a break after .8 hours

*** 11 more are complete: 1498-1518 are ready for unbanning. ( .4 hours for the second half)

(th)

Last edited by tahanson43206 (2021-01-22 11:40:39)

SpaceNut · 2021-01-22 17:49:03

Group id's 1497 through 1518 have been unbanned and moved to user list....

tahanson43206 · 2021-01-23 09:02:14

For SpaceNut re ID's Recovery and Recruiting effort ...

There were no incoming messages in NewMarsMember * gmail.com this morning. An update was posted to the Slow Glass topic.

this post is reserved for today's batch 1519-1540

The count of Registered Members is up to 16,790

The count of pages of Not Verified ID's stands at 57. That count will decrease towards zero over the next Earth year.

The count of pages of banned spammer ID's stands at 71. That count will fluctuate as we convert them, and as you add new bans.

Eventually, in about three Earth years, that count will drop to zero.

In the meantime, today's batch is 22 ID's

***
Today's batch is ready for unbanning: 1519-1540

The count of Registered users is up to 16,812.

(th)

Last edited by tahanson43206 (2021-01-23 11:43:20)

tahanson43206 · 2021-01-23 10:39:25

This is primarily for kbd512, with a nod to SpaceNut to keep a watchful eye on developments ...

For kbd512 ... I took a look at the books available at my local computer store, on PHP and MySQL ... there were at least 5 in the selection ..

All but one were priced at $50 or so . I chose the one for $17.19 (tax included) by Mike McGrath ... (2nd Edition/2018)

McGrath's presentation is perfect for me at this point of inquiry ... I've been looking at the FluxBB PHP scripts, and at the MySql table definitions.

In addition, I had a chance to run the ID Recovery project by two very experienced professionals (albeit retired) at a recent online meeting.

I came away convinced that the risks of trying to perform a mass update of all the spammer ID's using a special-purpose PHP script are too great.

Even if the script were tested in a backup duplicate environment, there is still a major risk of implementing something like that on the production forum database.

However, from previous experience, long ago and far away, I know that keyboard interactions with a web site can be handled by a program.

In thinking about what such a program might look like, I am imagining a capability to launch a set of code for each ID to be recovered.

In recent times, I've been experimenting with accomplishing the various tasks using only the keyboard, and have concluded that the entire set of steps can be performed using keyboard actions, and therefore that those same steps can be carried out by a program.

In practice, were such a program in operation, the operator would position the system at the two NewMars web pages needed, and touch a key to launch update of one ID to be recovered. The process would flow at the rate of the Internet connection. At the conclusion, the screens would show the completed transaction. The operator would visually verify success of the transaction, and (assuming all is well) activate the next sequence.

For you: Can you suggest a book or an article (or perhaps a YouTube video?) which covers design of a program to interact with a web site in the step by step manner described above?

Thanks!

Edit#1 ... this is an afterthought but perhaps Chromium itself might provide the programming environment suitable for the update sequence described above.

At the local computer user group (before the current virtual environment) I attended presentations showing how the browser (and Chrome in particular) can assist a developer in a variety of ways. At the time I just noted the capability, but now it occurs to me it might be a good fit for the present opportunity.

Edit#2: the copy of Chrome running on this (ancient) work station is: Version 27.0.1453.110

The current 64 bit version of Chromium is 88.0.4324.96

Edit#3 ... Found this ...

Programming Chrome Apps PDF Download for free:
This book appears to be available in hardcopy form from O'Reilly
Book Description:
Put your web app design skills to work by learning how to create powerful and portable Chrome Apps. With this practical book, you’ll learn how to build Google’s unique apps to behave just like native apps so they can interact with hardware devices, access external files, and send notifications.
Author Marc Rochkind takes you through a hands-on, objective tour of Chrome Apps, which run on any platform that supports the Chrome browser—including OS X, Windows, Linux, as well as Android and iOS. If you know how to work with HTML, CSS, JavaScript, and the DOM, you’re ready to get started.
Learn how to build, run, and debug Chrome Apps step-by-step <<== seems right
Use Chrome Apps to access local files, sync files, and external files <<== Yup
Take advantage of key-value-pair APIs, including sync storage and IndexedDB
Use WebSockets, Google Cloud Messaging, and other networking methods <<== seems right
Display graphics and images with Canvas, SVG, and the Media Galleries API
Use alarms, context menus, location, the camera, Bluetooth, USB, and other APIs
Publish apps to the Chrome Web Store with the Chrome Dev Editor <<== not for ** this ** application

Edit#4 ... There you go! The specific steps I've been carrying out are documented earlier in this topic.

However, I did not provide a tag for them to make it easy to find them. I'll have to come back to this later.

(th)

Last edited by tahanson43206 (2021-01-23 11:28:52)

SpaceNut · 2021-01-23 20:51:06

unban performed on id's 1519 through 1540 with them being moved to the user list

tahanson43206 · 2021-01-24 08:37:06

For SpaceNut re #720

Thanks for taking care of the latest batch of ID's! The count of Registered Users is up to 16,812. It should exceed 19,000 in a few months.

This post is reserved for today's progression, from 1541 through 1562

***
For SpaceNut re investigation of use of Chrome/Chromium as a programming environment ...

I ordered a book about using Chrome/Chromium as a platform to develop an app. It'll arrive some time in February.

In the meantime, I just turned on the trace feature in the Developer Tools kit, and watched as GET and POST commands flowed between the workstation and the server. Apparently there is a logging service (which I have not yet learned how to activate). That would provide the data needed to create a program to perform ID Recovery automatically. As reported in an earlier post, I am thinking about automating the processing for a single ID, so that the activity of the program can be monitored and checked closely by the operator to be sure it completed correctly and did not cause damage to any other data in the database.

***
ID's 1541-1562 were completed in a single sequence over 1.1 hours

That time included study of the activity log built into Chrome.

(th)

Last edited by tahanson43206 (2021-01-24 13:58:22)

SpaceNut · 2021-01-24 20:24:47

1541 through 1562 all done and moved

The php log files look like the pieces that I posts a while ago....

Took a peak at the number of users just a few moments ago and it was Guests online: 131
Those would be spammers are trying to create still....
evil laugh
ha ha no spam for you …

edit
converted another group ids 02051 - 02060

tahanson43206 · 2021-01-24 20:51:05

For SpaceNut re #722

Thank you for taking care of 1541 - 1562 !!

Regarding the spammers ... there are still thousands of accounts created by spammers and successfully registered, so your campaign of banning is worth continuing. Still, closing off registration has clearly made a big difference.

I suspect our Medical Student who stopped by to inquire about membership may have decided the essay question admission hurdle was too much for now.

It seems to me we don't need to hurry, and we don't need to lower the entry hurdle. There are 7 billion people on Earth ... the ones who would be able to contribute are alive and in school or early career right now.

If you've had a chance to look at the rover competition videos, you have seen the caliber of people who are alive and active on Earth right now.

(th)

tahanson43206 · 2021-01-25 07:11:22

This post is reserved for ID's 1563-1585

The first half are finished after .5 hours. Time for a break.

1586-1584 are completed. I changed the name of the Banned ID after the Test series so I could find it when verifying the updates.

***

It's been cold and wet, and not ideal Suburu engine working conditions.

(th)

Last edited by tahanson43206 (2021-01-25 11:59:39)

SpaceNut · 2021-01-25 17:56:21

Found 1563 -1584 to unban and move to users list.

New Mars Forums

Announcement

#701 2021-01-18 18:36:44

Re: Housekeeping

#702 2021-01-18 18:49:39

Re: Housekeeping

#703 2021-01-19 09:43:27

Re: Housekeeping

#704 2021-01-19 18:44:45

Re: Housekeeping

#705 2021-01-19 19:57:31

Re: Housekeeping

#706 2021-01-19 20:19:51

Re: Housekeeping

#707 2021-01-19 21:44:07

Re: Housekeeping

#708 2021-01-20 08:15:29

Re: Housekeeping

#709 2021-01-20 08:16:19

Re: Housekeeping

#710 2021-01-20 20:49:11

Re: Housekeeping

#711 2021-01-20 21:16:03

Re: Housekeeping

#712 2021-01-20 21:23:58

Re: Housekeeping

#713 2021-01-21 10:33:15

Re: Housekeeping

#714 2021-01-21 19:01:04

Re: Housekeeping

#715 2021-01-21 19:50:51

Re: Housekeeping

#716 2021-01-22 06:52:48

Re: Housekeeping

#717 2021-01-22 17:49:03

Re: Housekeeping

#718 2021-01-23 09:02:14

Re: Housekeeping

#719 2021-01-23 10:39:25

Re: Housekeeping

#720 2021-01-23 20:51:06

Re: Housekeeping

#721 2021-01-24 08:37:06

Re: Housekeeping

#722 2021-01-24 20:24:47

Re: Housekeeping

#723 2021-01-24 20:51:05

Re: Housekeeping

#724 2021-01-25 07:11:22

Re: Housekeeping

#725 2021-01-25 17:56:21

Re: Housekeeping

Board footer