You are not logged in.
For SpaceNut re Post Repair initiative ...
Today's Trail Breaker will try for 27600
Today's Re-scan will try for 22100
218,245,650 km [25.2 km/s] 9 AM local time
***
This next section is an attempt to document and to start an attempt to understand what happened during the WBA update of a very long post #27043.
The log shows that everything looked normal to WBA. The post was downloaded, processed locally to remove bandits, and submitted to FluxBB for posting.
What happened next started out normally ....
FluxBB knew what post it was working on: 27043
It knew the author of the post, the time of original posting, and it's place in the topic....
Shortly into the delivery of data to the original post, something happened that caused FluxBB to "think" a submit had been issued.
FluxBB completed the update to post 27043, with the small amount of data it had on hand.
However, WBA had sent a total in excess of 20,000 characters...
The mystery resumes here... FluxBB had closed the post it was supposed to be updating.
Apparently it retained "knowledge" of the topic, and it knew who was sending data (my ID).
It appears to have decided to create a new post using my ID, and it pushed some of the arriving data into the post.
For some reason, it then decided to "submit" the data and it completed a fake post ... it did that three or four times.
What this seems to say is that the FluxBB programmers were bound and determined to use every byte that their customer uploaded, and not to give up control.
They appear to have programmed fallback procedures for ambiguous situations like this one.
***
Completed Sequence for ID: 27600
Special to SpaceNut .... nice summary: http://newmars.com/forums/viewtopic.php … 36#p191736
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 300
Starting Number: 27301
Last Number of Run: 27600
Summary for Web Automation Report for 03-02-2022 at 13:16:41
Average time of Loop from Main form: 00:00:37
12 Skip Exceptions were recorded.Total time of Processing: 03:06:03
01 Sequence for ID: 27359 img url
02 Sequence for ID: 27452 same
03 Sequence for ID: 27514 same
04 Sequence for ID: 27521 same
05 Sequence for ID: 27532 same
06 Sequence for ID: 27545 same
07 Sequence for ID: 27552 same
08 Sequence for ID: 27570 same
09 Sequence for ID: 27580 pothole
10 Sequence for ID: 27583 pothole
11 Sequence for ID: 27589 pothole
12 Sequence for ID: 27592 pothole
A re-scan covering 21301-22100 will start now ...
The Re-scan was setup with a goal short of the intended value
Completed Sequence for ID: 21600
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 296
Starting Number: 21305
Last Number of Run: 21600
Summary for Web Automation Report for 03-02-2022 at 17:28:24
Average time of Loop from Main form: 00:00:38
3 Skip Exceptions were recorded.Total time of Processing: 03:10:20
Sequence for ID: 21335 pothole
Sequence for ID: 21348 pothole
Sequence for ID: 21350 pothole
Re-scan from 20601-22100 will begin now. this 500 post run will take (about) 5 hours and finish after midnight local time.
Update next day: The re-scan to 22100 finished with no more skips.
(th)
Offline
Special item for SpaceNut ....
GW Johnson just provided an update to his Propulsion Study.
Since you had indicated you might have taken a look at an earlier version, please pull this one down onto a full sized computer so you can study it.
If you do not have a full sized computer or Internet connection, I am willing to work with you to correct that situation.
The method I have in mind is working with Executive Director James Burk to allocate memberships to support of Mars Society staff.
In order for this to work, the position of Senior Forum Administrator would need to be defined as a part time volunteer position to which Society resources can be allocated.
If Internet service costs $50 (USD) per month, then only 12 full memberships in the Forum would cover that cost.
If a modern computer costs $1200, then an additional 12 full memberships would cover that one time cost.
We have over 1000 former members to contact by email, when you have time to think about doing that.
I have offered to write a script to send you a test email, and remain ready and willing to work on that mini-project.
(th)
Offline
I have the latest GW document up for reading.
The login popup box comes up when the link is opened but just close it and it stays up.
Offline
For SpaceNut re #2028... do you think GW's paper could be turned into a seminar?
As you pointed out, it contains procedures that are not familiar to many of us (ordinary humans).
(th)
Offline
For SpaceNut re posts repair ...
Today's Trail Breaker will try for 27900
I'm thinking of increasing the rescan from 800 to 900 to reach an even 23000.
Update at 10:57 local time...
Completed Sequence for ID: 27900
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 300
Starting Number: 27601
Last Number of Run: 27900
Summary for Web Automation Report for 03-03-2022 at 10:56:02
Average time of Loop from Main form: 00:00:38
4 Skip Exceptions were recorded.Total time of Processing: 03:14:02
Total time Program was Active: 03:21:10
Note: at least 22 unneeded tabs were created during this run ...
01 Sequence for ID: 27637 img url
02 Sequence for ID: 27668 same
03 Sequence for ID: 27694 pothole
04 Sequence for ID: 27859 img url
A Re-Scan of 900 posts (to 23000) will start now.
ETA is 23:18
The Re-scan finished a little before 22:00, well ahead of schedule!
Completed Sequence for ID: 23000
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 852
Starting Number: 22149
Last Number of Run: 23000
Summary for Web Automation Report for 03-03-2022 at 21:53:04
Average time of Loop from Main form: 00:00:41Total time of Processing: 09:42:02
Total time Program was Active: 09:42:15
Amazing! No skips ... there ** were ** some Internet episodes ... there are multiple tabs on the browser.
Those occur when the flow from WBA goes astray at the server and it responds with a display of the Index.
However, since no skips were reported, I'm assuming whatever the problem was did not have any effect on the run.
I'm planning a regular Trail Breaker of 300 posts tomorrow, and another 1000 for the Re-scan.
The re-scans seem to be going smoothly, so I'm encouraged to try a bit longer run to catch up sooner.
(th)
Offline
GW stuff is being collected in many posts but with it ending up in the orbital mechanics we are getting the information required to do so.
I see kbd512 was active with a bit of censoring but it was just the usual not being able to hold back on name calling.
Mars_B4_Moon was also a busy poster as well.
Offline
For SpaceNut re #2031 .... A Re-scan to 23000 just ended ... I'll be checking the log shortly...
In the meantime, thanks for noting kbd512's intervention in the dispute reported by Terraformer.
I liked the way he handled that! I thought it was adroit ... Clark is very different from the man I am seeing 20 years ago, but even back then he was acerbic. Now he seems to try to pack a lot of sting into as few words as possible.
You're certainly right about Mars_B4_Moon ... I don't have time to read all his posts, but I ** do ** appreciate his bringing old topics back with up-to-date connections to current events.
Just FYI, and assuming you are the only one who will ever bother to look at Housekeeping.... I am worried about some recent episodes of Antisemitism mixed in with white chauvinism. I'm trying to encourage talented people to join this endeavor, and having such posts in the mix would drive most young people of today away.
It may well have already happened, by the absence of applications, and the silence of those who have joined.
***
Looking forward to your adventures in finding a solution to the Radiation problem. I'll try to catch up on posts tomorrow.
(th)
Offline
I posted a little more about getting the rf and magnetic field to high as to the danger and damage it can do to a person long term.
Currently I want to delve into the food, water and air requirement mass without recycling and then with RobertDych has time as well to add to that information.
I am still reading the orbital mechanics post and looking hard at the space tug and whether we can send a unit to mars ahead of schedule but still have the fuel load we need to return after the 500 day stay at mars....
Offline
For SpaceNut re #2033
Those are all worthy endeavors! Thanks for taking them on!
If you spread yourself too thin, there is a risk you'll miss the opportunity to accomplish anything in any one area.
In a system of specialization, individuals focus upon narrow fields in order to advance the understanding of the group.
RobertDyck has given himself overall responsibility to integrate inputs from many people to create a unified whole. Those of us who are trying to help can work on parts of the puzzle and try to advance our part, because (of course) RobertDyck can't do all that detail work himself.
If RobertDyck needs help with specializations like food, water, air and recycling, then we need to find individuals who will be willing to concentrate on those.
Eventually this project will require thousands of people, and quite likely this will become a global enterprise.
I am taking one trillion US dollars as a ballpark figure for expense.
I'm basing that on 1 billion US dollars per passenger, in order to put an arbitrary target up so that an audience can get a sense of the size of the problem to be addressed.
GW Johnson has provided a target mass of 5000 tons. That was done to facilitate computations of fuel requirements for a flight to and from Mars, as you have noted. I am now offering a trillion dollars as a arbitrary financing target for a similar reason. It will be clear that this is a global challenge, and folks who want to participate will be thinking in those terms.
(th)
Offline
For SpaceNut re posts repair initiative ...
A Trail Breaker to 28200 completed with a Green Screen
Completed Sequence for ID: 28200
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 300
Starting Number: 27901
Last Number of Run: 28200
Summary for Web Automation Report for 03-04-2022 at 11:35:29
Average time of Loop from Main form: 00:00:36
3 Skip Exceptions were recorded.Total time of Processing: 03:00:54
Skip Details:
01 Sequence for ID: 27907 img url
02 Sequence for ID: 27946 same
03 Sequence for ID: 28162 same
A re-scan to 24000 is up next: ETA 23:00 local time
The rescan over 1000 posts generated no skips.
Completed Sequence for ID: 24000
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 1000
Starting Number: 23001
Last Number of Run: 24000
Summary for Web Automation Report for 03-04-2022 at 23:50:43
Average time of Loop from Main form: 00:00:40Total time of Processing: 11:10:37
Tomorrow's Trail Breaker will try for 28500.
Tomorrow's Rescan will try for 25000.
(th)
Offline
For SpaceNut re Post Repair initiative ...
A run to 28500 is underway.
I am planning to start the Rescan after today's Zoom meeting. It will run over 11 hours, so will end up in early next morning.
***
New subtopic ... I am sure no one else will pay any attention to this boring topic ...
I am ** really ** encouraged by the recent work of GW Johnson....
I'll be going back to the just-published document this weekend, to be sure, but I ** think ** I see a solution to the all-Chemical, Tugs-at-Earth problem.
GW Johnson himself has gone with Tugs at both ends as a near optimum solution;
It appears (subject to confirmation by looking back at the document) that 21,000 tons of propellant is the "magic" number ...
If I am remembering the paper correctly, and if I ** understood ** it correctly, 21,000 tons of propellant is what the Large Ship of 5000 tons needs to leave Mars and to perform minor course correction on the way back to Earth.
Perhaps by coincidence, it would appear that 21,000 tons of propellant is what Large Ship needs to reach escape after Tug release, and to enter LMO.
That tells me that 21,000 tons is needed in orbit at Mars, to restock the Large Ship with propellant for the flight home.
That 21,000 tons could come from facilities on Mars, but in the beginning those facilities will not be available.
Instead, the 21,000 tons could be placed in orbit at Mars by tankers from Earth.
If we had four tankers with 5000 tons of propellant each, they would total to 20,000 tons.
Each of those tankers would require 21,000 tons to reach escape and to achieve LMO.
The grand total for the fleet would be 5 times 21000 or 105,000 tons pushed off to Mars by the Earth tugs.
If my reading of GW's paper is correct, 105,000 tons of propellant is the magic number to enable one Large Ship with passengers and crew to fly from Earth to Mars and back, assuming both Departure and Arrival tugs are in the mix.
The 105,000 tons must all be shipped up from Earth at this point, but GW Johnson has provided figures for that.
It should be possible to arrive at a Grand Total of propellant needed for an all-Chemical trip plan with Tugs at Earth for Departure and Arrival.
That is the number I am looking for.
It is past time for hand waving. I am looking for firm estimates so everyone involved can understand the scope of the problem to be addressed.
***
Tying up loose ends ... The four tankers could be docked at Phobos for use as part of a service station there, when Mars is able to supply propellant.
It seems to me quite likely that the tankers could be clusters of Starships. They don't need rotation on their way to Mars. All they need is deep space shade to prevent boiloff. James Webb engineers have shown how to prevent boil off.
***
More loose ends ... A fleet of 21 Starship tankers could re-provision Large Ship at Mars. They would need just enough fuel to dock at Phobos.
They would then be a resource of considerable value, waiting for their owner to put them back into service at Mars.
Update at Noon local time... the Trail Blazer reached 28500 with a Green screen...
Completed Sequence for ID: 28500
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 300
Starting Number: 28201
Last Number of Run: 28500
Summary for Web Automation Report for 03-05-2022 at 11:56:56
Average time of Loop from Main form: 00:00:36
9 Skip Exceptions were recorded.Total time of Processing: 03:02:26
9 skips ... probably img url issues
1 Sequence for ID: 28360 img url
2 Sequence for ID: 28393 same
3 Sequence for ID: 28394 same
4 Sequence for ID: 28396 same
5 Sequence for ID: 28403 same
6 Sequence for ID: 28404 same
7 Sequence for ID: 28405 same
8 Sequence for ID: 28407 same
9 Sequence for ID: 28478 same
For SpaceNut re img / url problem ... there appear to be enough of these to justify investing some time in automation. I'm thinking of a small local utility that would work with the clipboard.
I might even be able to put something together that you could run as an Administrator.
I'll describe how this would work, and invite you to think about it...
Let's say that we (you or I) are looking at a post that contains bandits...
Currently, we dive into the edit on our browser screen, clean up the post, and submit the revision. If FluxBB likes our work, it accepts the offering and stores it in place of the original post.
We could do something similar ... we could highlight the text in the edit window with Ctrl/A and copy it to clipboard with Ctrl/C
We could then click on a utility program.
The utility program that i am imagining would:
1) Copy the clipboard contents to internal memory
2) Perform all required cleanup in memory
3) Write the revised text back to the Clipboard
This would all happen in the blink of an eye.
The operator would then click back on the edit window in the browser.
This time, the operator would enter Ctrl/A to highlight the text, and Ctrl/v to paste the revised text.
The operator ** should ** see the corrected text in the browser edit window.
The operator would then click Submit, and if FluxBB likes the result it would store the post over the old one.
I can easily imagine writing a utility for your Administrator task of preparing User ID's for service, although I have no idea what steps you carry out to perform the adjustment.
I do recall that kbd512 showed there is a way to automate the procedure, but for whatever reason he left the untreated ID's for you to finish.
***
As reported earlier, i'm going to run today's Rescan after today's special Zoom session. it will run for 11 to 12 hours, and will finish some time in the early hours Sunday. 25000 is the next target. This new procedure is gaining 700 posts on the Trail Breaker, compared to the previous gain of 500 posts. i am looking forward to catching up.
However, after catching up, I'm planning to perform the Rescan over the 300 posts just processed, to catch any errors that might occur. At this point, the Trail Breaker seems to be performing well, so the result of the Rescan should normally be a Green screen with zero skips.
Update at 15:15 local time, after Dress Rehearsal Zoom .... The meeting went well, in the sense that we (NewMars) are in position to cover March 12th if RobertDyck is not available due to his employment. He may be called away to some remote part of Canada at any moment. If that happens, Dr. Johnson will provide a short cover presentation, ahead of the full presentation on April 9th.
I have asked North Houston to try to set up opportunities for Dr. Johnson to meet folks who can help him to develop an online education experience for future space craft navigators. The North Houston rep has friend/acquaintance at Huntsville who may be able to help develop an online education package.
I'll start the Rescan from 24001 to 25000 now. I'm looking forward to having the rescans out of the way.
Update at 20:54 local time ... if you get this far into the post, thanks for providing the image of the Ukrainian homemade drone. I'd read a little bit about their effectiveness, but the additional details were interesting! The ability to fly without direct control was a surprise (to me for sure!).
Update next day ... a Re-scan over 1000 posts finished with a Green screen and 8 skips
Completed Sequence for ID: 25000
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 1000
Starting Number: 24001
Last Number of Run: 25000
Summary for Web Automation Report for 03-06-2022 at 06:54:01
Average time of Loop from Main form: 00:00:40
8 Skip Exceptions were recorded.Total time of Processing: 13:37:47
1 Sequence for ID: 24693 pothole
2 Sequence for ID: 24697 pothole
3 Sequence for ID: 24701 pothole
4 Sequence for ID: 24702 pothole
5 Sequence for ID: 24707 pothole
6 Sequence for ID: 24721 pothole
7 Sequence for ID: 24724 pothole
8 Sequence for ID: 24727 pothole
We seem to have a clean slate through 25000 - just under 1/3 of the way to 77,000
A new Trail Breaker will start shortly. Today's Rescan will try for 26000.
Catch up should occur in (about) fiv e Earth days.
(th)
Offline
For SpaceNut re Posts Repair initiative ...
Today's Trail Breaker will try for 28800. Today's Rescan will try for 26000.
The Trail Breaker finished with a green screen and 5 Skips
Completed Sequence for ID: 28800
Total Command Lines found: 44
Total input Lines in script: 140Number of ID's processed: 300
Starting Number: 28501
Last Number of Run: 28800
Summary for Web Automation Report for 03-06-2022 at 10:24:45
Average time of Loop from Main form: 00:00:35
5 Skip Exceptions were recorded.Total time of Processing: 02:55:14
Skips detail:
1 Sequence for ID: 28504
2 Sequence for ID: 28505
3 Sequence for ID: 28550
4 Sequence for ID: 28684
5 Sequence for ID: 28732
I'll start the Re-scan now while looking at possible automation of the img/url problem
Update at 11:50 local time ... After a bit of study, I decided to try adding new script lines to the existing script.
The new lines are:
#
# Adding script to remove url inside img elements 2022/03/06
PerformFunctiondeleteTextClipboardZero:[ u r l ]
PerformFunctiondeleteTextClipboardZero:[ / u r l ]
PerformFunctiondeleteTextClipboardZero:u r l =
#
I tested them on the five skips reported above, and they seemed to work (a) and (b) not cause other problems.
I'll resume the rescan and try the new script over 1000 previously processed posts .. Update at 23:30 local time ... the new commands did not cause any problems over the range 25001-25823...
Update at 23:10 after Zoom meeting ... I broke the Re-scan into two chunks, to avoid bandwidth concerns during the Zoom. The first phase, from 25001 to 25823, found 5 Skips ...
1 Sequence for ID: 25386 pothole
2 Sequence for ID: 25387 same
3 Sequence for ID: 25407 same
4 Sequence for ID: 25410 same
5 Sequence for ID: 25412 same
I'll start the finish stretch to 26000 now...
Update next day ... The Rescan to 26000 finished with a Green screen ...
Completed Sequence for ID: 26000
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 177
Starting Number: 25824
Last Number of Run: 26000
Summary for Web Automation Report for 03-07-2022 at 07:10:11
Average time of Loop from Main form: 00:00:36Total time of Processing: 01:48:23
There were no skips .... Today's run's will start shortly.
(th)
Offline
For SpaceNut re Forum Archive maintenance ...
I've started work on a Clipboard Update Clipboard utility program.
It would (if I complete it) read a script in using the clipboard, and then process text that is stored in the Clipboard.
It would update the text according to the script, and then write the results back to the clipboard.
All the operator would have to do is to highlight text, copy it to the clipboard, click a button and paste the clipboard back to the source location.
There's no reason that system wouldn't work, because that is precisely what WBA is doing, except that WBA pulls it's instructions using a text file input process.
My heart isn't in this, because I have no reason to think you would use if if I wrote it.
Your comfort level with manual editing of all those hundreds of posts from 2004 on is so high, that my guess is you'd rather do that and not use a utility.
(th)
Offline
For SpaceNut re #2038
It is possible I might be able to implement the text repair Utility as a web resource.
I'm not sure (at this point) how such a program would work, but if I ** were ** able to create the Clipboard Update Clipboard utility as a web resource, would you use it?
The shell of the new program is finished ... It has buttons, a text box and a text work space.
The buttons are:
About (explain program and operation)
Fetch Clipboard - copy clipboard text (only text) and paste it into work space
Load Script - must be done using script copied to clipboard from text source file
Run Script - requires the operator to fetch the text to work on
Copy to Clipboard - Done after operator visually confirms the requested work was done correctly
End - End program
If the operator has more than one post to repair, then the cycle would be:
1) Set up program (or call from web if it is available there
2) Open script file in Notepad or equivalent
3) Select all and copy it to clipboard
4) Button: Fetch Clipboard - the script will appear in the work window
5) Button: Load Script - program will examine the script and load valid instructions
6) Open file/object with text to be corrected (eg, post on NewMars)
7) Select all and copy it to clipboard
8) Button: Run Script - examine result
9) Button: Copy to Clipboard - assumes text is correct - otherwise, fix it on the work screen
10) At target location, paste revised text in place of original
11) At target location, Submit the revised post
Repeat until posts are all repaired or operator is out of time
(th)
Offline
For SpaceNut re Posts Repair initiative ...
Today's Trail Breaker finished with a Green screen...
Completed Sequence for ID: 29100
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 300
Starting Number: 28801
Last Number of Run: 29100
Summary for Web Automation Report for 03-07-2022 at 11:23:02
Average time of Loop from Main form: 00:00:35Total time of Processing: 02:58:05
There were no skips! Nice!
I'll begin today's Rescan to 27000 shortly
Update at 22:53 local time ... the Rescan just ended with a Green screen and no skips.
Completed Sequence for ID: 27000
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 1000
Starting Number: 26001
Last Number of Run: 27000
Summary for Web Automation Report for 03-07-2022 at 22:52:22
Average time of Loop from Main form: 00:00:40Total time of Processing: 11:10:19
Total time Program was Active: 11:10:26
EOJ Msg#1: Successful Run. No Warning Errors.
Tomorrow's Trail Breaker will try for 29400.
The Rescan will attempt another flawless inspection of 1000 posts, to 28000.
If the current schedule holds, both runs will reach 30000 on the same day.
(th)
Offline
For SpaceNut ... this item could go into multiple topics ...
There is something for practically everyone, from Louis (for solar power) to Dr. Zubrin, for making methane with hydrogen).
https://www.msn.com/en-us/news/technolo … hp&pc=U531
Popular Mechanics
This Green Hydrogen Plant Could Soon Fuel SpaceX Rockets
Jennifer Leman - 26m agoEnergy startup Green Hydrogen International (GHI) is planning to develop a new project based in South Texas.
The hub plans to supply billionaire Elon Musk’s SpaceX with green hydrogen for the rocket company’s methane-powered rocket engines.
Other uses for the abundant, colorless gas include adding it to ammonia for fertilizers and generating environmentally friendly jet fuels.
Mars evangelist and SpaceX founder Elon Musk may soon have an easier time going green.Green Hydrogen International (GHI), a Texas-based energy startup, announced plans last week to develop a brand-new green hydrogen project—the world’s largest—in South Texas. GHI says some of the hydrogen from this project will be used to develop clean rocket fuel for Elon Musk’s aerospace company SpaceX.
You love energy. So do we. Let’s nerd out over it together.
When completed, the staggering 60-gigawatt project—located near a hydrogen storage facility in Duval County, Texas—will produce over 2.5 billion kilograms of green hydrogen per year. (The largest green hydrogen plant in the world, Air Liquide’s 20-megawatt Bécancour facility in Quebec, Canada, produces just 300,000 kilograms of hydrogen per year.) A mix of solar and wind energy will keep the facility’s lights on, and during low-price periods, the Electric Reliability Council of Texas (ERCOT), the state’s electrical grid, will also supply power.Continue reading
(th)
Offline
Musk has realized that a high launch rate means the supply will be a problem from local sources and its going to require a different one in order to fly this ship as often as not only he would like but for what we are planning as well.
Offline
For SpaceNut re Post Repair initiative ...
A Trail Breaker is in progress... target is 29400
Today's Rescan will try for 28000.
Re #2042 ... thanks for noting the proposal to build a solar and wind power Hydrogen production facility in Texas.
Completed Sequence for ID: 29400
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 300
Starting Number: 29101
Last Number of Run: 29400
Summary for Web Automation Report for 03-08-2022 at 11:27:16
Average time of Loop from Main form: 00:00:36Total time of Processing: 03:00:18
No skips were recorded.
The Rescan to 28000 will start now.
WBA was updated to report if a timeout occurs. The program may not be reporting all skips.
<glum> Confirmed ... the program was NOT reporting all skips ... It ** was ** reporting those that dealt with expected situations, but NOT those that resulted from unexpected situations that previously caused halts.
The Rescan will include writing to the log if a Timeout occurs. The ** next ** version will include Skip count update.
It would appear that the entire set of posts processed previously must be re-scanned.
I'll finish the current catchup process before starting all over again.
In then end, we should have nothing in the log to report except potholes.
Update a bit later ...
The problem is in this section of script:
ReportClipboard:0
PasteClipboard:0
wait 1000
tab:3
#
<esc>enter.
Wait 1000
#
# The record should have updated
The fact is, as 27043 shows, one second is not enough time for the upload of a long post.
The submit is transmitted before the text has been delivered to the post, and the submit is being taken out of order. This would make sense if the post is 20,000 characters, and packets hold 512 ... the packets travel over different paths, and while they are presented to the destination in sequential order, the submit is a ** different ** packet which is not part of the long post.
I'll have to look at this situation, to see if there is anything transmitted by FluxBB that confirms an upload has finished.
In the meantime, I'll resume the rescan at 27075...
Update at 13:23 after preliminary failure analysis ...
The command that uploads data after editing is Paste from Clipboard ...
There is a one second wait and the wait is followed by an Enter.
That is the point of failure ... the vast majority of posts complete upload during the one second delay.
A 20,000 character post simply needs more time. At the point just before the Paste from Clipboard, the size of the post is known.
It ** may ** be possible to introduce a variable delay, such as one second for each 512 characters.
For 20,000 characters, that would be a delay of 40 seconds .... To prevent this kind of failure, that delay seems reasonable.
I'll look into the possibility of a variable delay tomorrow.
There ** is ** one other possibility ... I notice that there is a line written to the address bar after submit, but I'm not sure if it is useful.
It ** may ** occur when submit arrives, regardless of the status of the data flow.
Update at 23:25 local time ... the Rescan to 28000 just finished, with a Green Screen ...
Completed Sequence for ID: 28000
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 926
Starting Number: 27075
Last Number of Run: 28000
Summary for Web Automation Report for 03-08-2022 at 23:26:02
Average time of Loop from Main form: 00:00:40
15 Skip Exceptions were recorded.Total time of Processing: 10:18:12
Total time Program was Active: 10:22:06
15 skips were reported ....
Details to follow...
01 Completed Sequence for ID: 27216 Time Out! Corrected
02 Completed Sequence for ID: 27217 Time Out! Corrected
03 Completed Sequence for ID: 27277 Time Out! Corrected
04 Completed Sequence for ID: 27580 pothole
05 Completed Sequence for ID: 27583 pothole
06 Completed Sequence for ID: 27589 pothole
07 Completed Sequence for ID: 27592 pothole
08 Completed Sequence for ID: 27694 pothole
09 Completed Sequence for ID: 27910 Time Out! Corrected
10 Completed Sequence for ID: 27985 Time out! Corrected
The count of potholes was doubled in this version. i'll fix that tomorrow.
I'll also take a look at the Submit timing problem ...
(th)
Offline
For SpaceNut re Post Repair initiative ...
A Trail Breaker is in progress... target is 29700
Today's Rescan will try for 29000.
WBA was corrected to avoid double counting of potholes.
I am still thinking about how to implement a revision to better handle long posts that take longer than one second to complete.
I would prefer a solution that goes into effect only to handle long posts such as 27043. The vast majority (ie, 29100 less one) of posts do not cause the problem.
Update at Noon+ Local time:
Completed Sequence for ID: 29700
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 300
Starting Number: 29401
Last Number of Run: 29700
Summary for Web Automation Report for 03-09-2022 at 12:34:54
Average time of Loop from Main form: 00:00:35Total time of Processing: 02:55:32
Surprise! There were no skips!
The Rescan to 29000 will start now and run for 11 hours or so...
Update at 14:49 local time ... the rescan is running, and there are no signs of problems so far. This version contains a correction to NOT double count potholes.
A new version is ready and it will be installed for tomorrow's runs ... this version contains a variable time delay in the Paste function. It will wait one second for every 512 characters in the post... that is probably overly cautious, and I may adjust that value if it generates too many delays. On the other hand, it ** should ** handle 27043, and I'm planning a run targeting ** that ** post to see what happens.
Update at 22:10 local time ... The rescan was stopped at 28596 to free up bandwidth.
1
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 28125 corrected
2
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 28274 corrected
3
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 28276 corrected
4
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 28326 corrected
5
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 28434 corrected
I'll start the finish stretch of the rescan now...
(th)
Offline
For SpaceNut ... This is a special post .... just FYI ...
We have been slowly adding topics as spinoffs of and/or in support of the original Large Ship topic.
Here is a list before today's addition of the Physical Fitness topic.
Sticky: Large Ship Chemical Propulsion by tahanson43206 [ 1 2 3 ]
Interplanetary transportation 50 2022-03-07 22:59:21 by SpaceNutSticky: Large Ship Cabin Design - Manufacture - Testing - Operation by tahanson43206
Interplanetary transportation 2 2022-03-06 11:02:23 by tahanson43206Sticky: Large scale colonization ship by RobertDyck [ 1 2 3 … 49 ]
Interplanetary transportation 1,203 2022-03-05 08:48:41 by SpaceNutSticky: Large Ship Radiation Protection by tahanson43206 [ 1 2 3 ]
Interplanetary transportation 51 2022-03-05 08:36:23 by SpaceNutSticky: Large Ship hull material by RobertDyck
Interplanetary transportation 21 2022-03-02 22:08:01 by kbd512Sticky: Large Ship Competing Designs by tahanson43206 [ 1 2 ]
Interplanetary transportation 46 2022-03-01 20:07:52 by SpaceNutSticky: Large Ship Orientation Navigation Pointing by tahanson43206
Interplanetary transportation 9 2022-02-27 08:45:40 by RobertDyckSticky: Large Ship Momentum (Mass) Management by tahanson43206
Interplanetary transportation 8 2022-02-26 08:31:01 by tahanson43206Sticky: Large Ship Atomic Propulsion by tahanson43206
Interplanetary transportation 7 2022-02-23 10:39:55 by tahanson43206Sticky: Large Ship Food Production Storage Recycling by tahanson43206
Interplanetary transportation 2 2022-02-23 07:11:05 by tahanson43206Sticky: Large Ship Food Preparation and Delivery by tahanson43206
Interplanetary transportation 7 2022-02-22 21:20:22 by SpaceNutLarge Ship Engineering Model by tahanson43206 [ 1 2 ]
Mars Analogue Research Stations 29 2022-02-22 19:56:57 by SpaceNutLarge ship analog by SpaceNut [ 1 2 3 ]
Mars Analogue Research Stations 58 2022-02-19 21:34:34 by SpaceNutLarge Ship Prime is: Large scale colonization ship by RobertDyck
The idea is to allow information that has accumulated in a vertical flow over two years to appear in subtopics that focus on a part of the overall concept.
I am trying to prepare in case RobertDyck's talk this Saturday generates some interest.
It's impossible (for me for sure) to guess what will happen. The default, based upon NewMars, is not much, but I'm prepared to be pleasantly surprised.
(th)
Offline
For SpaceNut re Post Repair initiative ...
I have bad news and good news .... I forgot I'd let the laptop run over night to reach 29000, and pushed the power button this morning instead of the space bar to wake it up. The result was loss of the End of Job report by WBA.
However, the log was intact, so a bit of searching provided the good news:
The new Delay for Long Post feature is clearly working ...
What I find surprising is how many posts were given a delay ... 412 out of 1000 got at least one second.
The maximum delay found was 11 seconds. if this was a typical run, that means that the default one second delay has been handling the entire load until now, except for 27043.
More bad news: The delay did not fix the problem we are seeing with 27043 ...
Today is catch-up day ... the Trail Breaker will try for 29000, and the rescan will do the same.
Because of improvements made in the script and in the program since we started repairs, I am tentatively thinking about rerunning the rescan from the beginning, starting next month.
My idea is to keep things simple by scanning 1000 posts from the beginning, based upon the day of the month. The run will return to 30,000 on April 30th, if I decide to pursue this.
Meanwhile, the Trail Breaker will keep chugging along at 300 posts per day, but it is now using all the improvements added in recent times, so there is less chance of a defect lingering in the archive.
Today's Trail Breaker will start shortly.
Update at 9:31 local time ... The Trail Breaker is running...
I decided to try the new version of WBA with 27043 ... failure occurred again.
After study of the log, I don't see a hint about what is happening,
so have decided NOT to try re-running the edits from the beginning.
There are a few posts left uncorrected, and it is possible they may be discoverable without using Edit mode.
Update a bit later .... The need for the re-scan remains ... the problem is with UPDATE. We started this entire process with scans that did not update.
I've revised my concept to perform the scan-without-update again.
The log will report any posts that contain bandits. It will be easy to find them.
Most daily runs over 1000 posts will contain no deletes at all ...
Update at 10:12 local time ... The failure observed with 27043 is (probably) NOT happening on the Internet ... I now suspect it is happening in the laptop ... I'm using a 4800 baud serial line between the WBA program and the browser ... I now suspect the I/O processes happening in multiple programs are overwhelmed... if this theory is correct, it (should) be possible to regulate the flow over that connection. In the mean time, the system seems to be working except for 27043, and I'm planning not to pester 27043 again, so further work may not be needed.
However, this ** is ** a reminder that the option to perform data transfers using software is available. It was faster to go with the hardware solution, so that is what is in service.
Update at 11:07 local time .... In watching WBA going about it's work, it came to me that there is more involved here than the exchange of data from WBA and the browser, which certainly seems to be the place where the failure is occurring. The data from WBA is delivered to the browser, which loads it up into the edit window, where (if a person were operating) it could be viewed before submit.
There ** is ** a way for WBA to perform that verification, and it does not require further traffic through the serial cable.
WBA ** could ** request the contents of the edit window, and compare what it finds to what it sent.
That wouldn't be too much of an advance, and it could trigger a halt if there is a mismatch.
One definite advantage of this extra processing step is that it would narrow down the location of the failure ... right now, the failure could be occurring on the laptop, or it could be occurring in the Internet side. If the text in the edit window of the browser is known to be the same as the text output from the edit, then that leaves the Internet side as the place to look
It might even be possible to implement this idea with the existing script commands.
(th)
Offline
For SpaceNut re Post Repair initiative ...
This post is dedicated to today's Catch up runs ...
A run to 30000 is underway ... a rescan to 30000 will be scheduled around noon local time.
Update at 12:48 local time ... Trail Breaker just reached 30000 ... a mini-Milestone ...
Completed Sequence for ID: 30000
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 300
Starting Number: 29701
Last Number of Run: 30000
Summary for Web Automation Report for 03-10-2022 at 12:46:20
Average time of Loop from Main form: 00:00:40
7 Skip Exceptions were recorded.Total time of Processing: 03:21:12
Skip detail to follow ...
Skip report:
1
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29704 << no problem found
2
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29797 << no problem found
3
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29917 << corrected
4
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29930 << no problem found
5
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29934 << corrected
6
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29935 << no problem found
7
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29956 << corrected
A new set of problems has appeared ... the log is showing there is nothing to work on ...
The script uses tabs to get to the edit window ... if one of those is ignored, then nothing (or the wrong thing) would be collected ... something ** else ** to look at.
The Catch Up Rescan will start now>>>
Update at 13:43 local time ... the rescan to 30000 is under way ...
The strange failures reported above seem likely to have been caused by Internet congestion ... I found congestion when I ran ping, so performed reset procedures until the congestion was cleared.
In addition, I brought a copy of the currently running script to the development PC, and set up a folder to work on it.
It seems likely this can be down without any changes to WBA ...
The point of failure appears to be the delivery of text from WBA to the Edit window in the browser.
This transfer needs to complete before Submit is transmitted.
The text copied to the Edit window is still in the variable where it ended up after editing.
It should be possible to select the edit window, copy the selected text to the clipboard, and compare the clipboard to the text in the output variable.
If the compare fails, the run should stop, or continue with a skip message.
Update at 19:00 local time ... the Rescan just completed 500 posts with 6 skips ...
I'll look at the skips now and restart the run at 29501.
Completed Sequence for ID: 29500
Starting Sequence for ID: 29501
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 500
Starting Number: 29001
Last Number of Run: 30000
Summary for Web Automation Report for 03-10-2022 at 18:56:35
Average time of Loop from Main form: 00:00:40
6 Skip Exceptions were recorded.Total time of Processing: 05:34:43
Skip report:
1
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29048 - TO FIX - OK
2
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29256 - TO FIX - corrected
3
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29384 - TO FIX - corrected
4
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29418 - MYSQL ERROR - ok now
5
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29470 - LOST - ok now
6
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 29494 - LOST - ok now
Starting the last 500 to reach 30,000 now ...
Update the next day ... WBA reached 30000 in rescan mode with a Green screen
There was one skip - the post updated without incident in manual model, so I interpret this as an Internet issue
Completed Sequence for ID: 30000
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 500
Starting Number: 29501
Last Number of Run: 30000
Summary for Web Automation Report for 03-11-2022 at 07:29:58
Average time of Loop from Main form: 00:00:39
1 Skip Exceptions were recorded.Total time of Processing: 05:22:28
(th)
Offline
For SpaceNut re posts repair initiative ...
Today's Trail Breaker will try for 30300, and the rescan will cover the same ground.
This pattern is planned through the end of March.
For April, I am tentatively planning to the 1000 rescan, but with a scan script that does not update.
That way we will avoid a problem with 27043, and (more importantly) pick up any posts missed earlier.
There should be very few remaining bandits, since every post has been visited twice, but the script has been improved along the way, and at one point skips were not being reported.
Update at 12:49 local time ... WBA generated a Red Screen processing post 30274...
Starting Sequence for ID: 30274
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 273
Starting Number: 30001
Last Number of Run: 30300
Summary for Web Automation Report for 03-11-2022 at 12:50:08
Average time of Loop from Main form: 00:00:38
3 Skip Exceptions were recorded.Total time of Processing: 02:55:47
Skip report:
Halt occurred at post 30274
Skip report:
1
Delay for long paste: 4 seconds
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 30048 - updated in manual mode
2
Delay for long paste: 2 seconds
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 30201 - updated in manual mode
3
Delay for long paste: 4 seconds
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 30273 - updated in manual mode
I wonder if Internet congestion was a factor. All three posts uploaded without a problem in manual mode.
The run to 30300 finished without further incident:
Completed Sequence for ID: 30300
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 27
Starting Number: 30274
Last Number of Run: 30300
Summary for Web Automation Report for 03-11-2022 at 13:28:52
Average time of Loop from Main form: 00:00:36Total time of Processing: 00:16:46
I'll now revise the script to Scan and Report instead of Update, and run that over the same 300 posts.
Update at 15:51 local time ... a new Report Only script just finished review of the 300 posts processed earlier.
Completed Sequence for ID: 30300
Total Command Lines found: 31
Total input Lines in script: 151Number of ID's processed: 300
Starting Number: 30001
Last Number of Run: 30300
Summary for Web Automation Report for 03-11-2022 at 15:50:51
Average time of Loop from Main form: 00:00:22Total time of Processing: 01:50:12
Without the update procedure, the posts take less time (of course)...
Never-the-less, I'm expecting them to show any left over touchup work that might be needed.
The need for touch up will be revealed by the presence of any occurrences of the text "deleted:" in the log.
For this run, there were no occurrences of "deleted:", so the previous run caught everything.
The laptop will get a longer break than usual, now that the 1000 post runs are history.
Update at 16:55 local time ... the improved performance of the Report-only-Rescan script implies that only 6 hours would be needed to review 1000 posts ... At that rate, I could imagine running all three runs in April:
1) Trail Breaker for 300 posts: 3 hours
2) Report-only-Rescan for the same 300 posts: 2 hours
3) Report-only-Rescan over 1000 posts: 6 hours
The total of 3+2+6 is 11, which is less than an update run against 1000 posts.
The laptop would ** still ** get a 12 hour break or so.
This strategy would allow confirmation that all bogies were handled in the first 30,000 posts, without causing further problems for poor old 27043.
(th)
Offline
For SpaceNut re Post Repair initiative ...
This is just FYI ....
The new Report-Only script performs the same operations on data from each post. The difference is that no update is attempted.
Command Code stored at Array line: 25 PFD Perform Function Delete in Row 0
Command Code stored at Array line: 26 PFD Perform Function Delete in Row 0
Command Code stored at Array line: 27 PFD Perform Function Delete in Row 0
Command Code stored at Array line: 28 PFD Perform Function Delete in Row 0
Command Code stored at Array line: 29 PFD Perform Function Delete in Row 0
Command Code stored at Array line: 30 PFD Perform Function Delete in Row 0Text to Delete Not found: color=#
Text to Delete Not found: color=
Text to Delete Not found: /color:
Text to Delete Not found: /color
Text to Delete Not found: :post_uid
Text to Delete Not found: []
If one of the strings shown above ** is ** found, then the report will show the number of characters deleted.
Because the first 30,000 posts were a learning period, there are stretches where a few bandits remain, so the rescan I am considering for April will (hopefully) report them, without disturbing the posts themselves.
One of us (Adminstrator/Moderator) can then (carefully) open the post in manual mode and carry out any corrections that might be needed.
I don't expect there will be very many such posts, but I'm sure there will be a few.
Meanwhile, the Trail Breaker will be chipping away at the 77,000 post total, 300 posts per day.
Update next day: I am thinking of calling the scan-and-report run The Inspector. It will go over the 300 posts just updated by the Trail Breaker.
The Inspector should find no flaws, so it would be a big deal if it found one.
This plan assumes that any posts skipped by the Trail Breaker will be attended to before the Inspector shows up.
(th)
Offline
For SpaceNut re Post Repair initiative ....
The Trail Breaker just reached 30600 with a Green screen and 2 skips.
Completed Sequence for ID: 30600
Total Command Lines found: 47
Total input Lines in script: 148Number of ID's processed: 300
Starting Number: 30301
Last Number of Run: 30600
Summary for Web Automation Report for 03-12-2022 at 11:11:09
Average time of Loop from Main form: 00:00:38
2 Skip Exceptions were recorded.Total time of Processing: 03:11:26
Total time Program was Active: 03:19:02
Skip report:
1
Delay for long paste: 1 seconds
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 30362 - corrected
2
Delay for long paste: 2 seconds
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 30450 << pothole
The new "Inspector" will start shortly. It will look at the same 300 posts and report any missed items.
Inspector will find the pothole. Hopefully that is ** all ** it will report.
Inspector finished with a Green screen ...
Completed Sequence for ID: 30600
Total Command Lines found: 31
Total input Lines in script: 151Number of ID's processed: 300
Starting Number: 30301
Last Number of Run: 30600
Summary for Web Automation Report for 03-12-2022 at 13:30:03
Average time of Loop from Main form: 00:00:23Total time of Processing: 01:56:56
There were no skips... there was one pothole ...
Now I'll look for issues ...
Now ** this ** is interesting! 30450 showed up as a pothole in the Trail Breaker.
The Inspector found it ...
30450 contained bandits, and they are removed. This episode seems to support the proposition that running the Inspector after the Trail Breaker is a good idea.
Tomorrow's runs will try for 30900. The laptop gets a nice long break!
(th)
Offline