You are not logged in.
For SpaceNut re ongoing activities...
Upgrade of TestID's is holding at TestID11200 still banned.
The proposal to send email to formerly active members is still pending.
Addition of a comment to the YouTube video of RobertDyck's talk is still pending.
Audit of the first 30,000 posts is still pending, with a start date of April 1, 2022.
(th)
Offline
For SpaceNut re image of PDP 11/70
http://newmars.com/forums/viewtopic.php … 91#p192491
I was stunned to see that image .... I spent at least ten years working in a team that programmed a pair of those in a regional bank. Those were the best years of a long period spent evolving with computers, from the 1401 at an Insurance company, through the latest handheld state-of-the-art equipment in 2000.
The team was doing interactive development long before our many IBM mainframe associates moved out of the punched card phase.
Thanks for that memory!
(th)
Offline
For SpaceNut re Post Repair initiative ...
WBA is modified to report the length of mismatched strings, when mismatch occurs.
Data is supposed to be transferred from the Edit output variable to the Browser window without error.
However, since that has been proven NOT to reliably be the case, in recent days, WBA has been reporting mismatches and skipping over the posts. It takes about five minutes to process each such post manually.
Today, I'm adding code to report the length of the text before and after transmission, as a first step to trying to understand the nature of the problem to be solved.
It is possible that a simple retry procedure may "fix" some of the problems.
Today's runs will try for an even number: 33,000.
Completed Sequence for ID: 33000
Total Command Lines found: 50
Total input Lines in script: 165Number of ID's processed: 300
Starting Number: 32701
Last Number of Run: 33000
Summary for Web Automation Report for 03-20-2022 at 16:27:35
Average time of Loop from Main form: 00:00:38
5 Skip Exceptions were recorded.Total time of Processing: 03:10:20
The 5 skips may include length data. The five skips reported in this run all have the same length.
Tomorrow I'll add code to find the differences, using a byte-by-byte compare process.
Skip report:
1
Delay for long paste: 2 seconds
Text received does not match sent. Skipping to Next Item
Count Sent: 1261 Count Received: 1261
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 32752 posted
2
Delay for long paste: 3 seconds
Text received does not match sent. Skipping to Next Item
Count Sent: 357 Count Received: 357
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 32773 posted
3
Delay for long paste: 3 seconds
Text received does not match sent. Skipping to Next Item
Count Sent: 1286 Count Received: 1286
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 32814 posted
4
Delay for long paste: 3 seconds
Text received does not match sent. Skipping to Next Item
Count Sent: 1603 Count Received: 1603
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 32832 posted
5
Delay for long paste: 3 seconds
Text received does not match sent. Skipping to Next Item
Count Sent: 1736 Count Received: 1736
Time Out! Skip Requested! Skipping to Next
Completed Sequence for ID: 32901 posted
Tomorrow's runs will try for 33300
If present plans hold, the WBA will report specific bytes that are different.
Today's Inspector will start now...
Inspector completed with Green screen and no errors discovered...
Completed Sequence for ID: 33000
Total Command Lines found: 31
Total input Lines in script: 151Number of ID's processed: 300
Starting Number: 32701
Last Number of Run: 33000
Summary for Web Automation Report for 03-20-2022 at 19:09:42
Average time of Loop from Main form: 00:00:22Total time of Processing: 01:49:17
(th)
Offline
For SpaceNut re Posts Repair Initiative
This post is reserved for today's runs to 33300... WBA will be updated to compare the bytes of sent/received text strings that do not match.
In yesterday's run, it was observed that the mismatched strings were the same length, so bytes are not being lost in transmission over the serial line.
However, it seems evident that bits are being lost, and chances are good that the missing bits are the high order bits of special characters.
Today's update will provide details about the location of the mismatch condition, and the format of the bytes sent and received.
The Trailbreaker finished with a Green screen and 18 skips ...
Each skip will take about five minutes, so I'm looking at nearly an hour of manual follow up.
However, this version of WBA ** should ** have reported differences it found between sent and received transmissions.
Completed Sequence for ID: 33300
Total Command Lines found: 50
Total input Lines in script: 165Number of ID's processed: 300
Starting Number: 33001
Last Number of Run: 33300
Summary for Web Automation Report for 03-21-2022 at 15:09:43
Average time of Loop from Main form: 00:00:38
18 Skip Exceptions were recorded.Total time of Processing: 03:10:05
Skip report before updates:
Skip report:
1
Delay for long paste: 7 seconds
Count Sent: 3542 Count Received: 33071
Completed Sequence for ID: 33037
2
Delay for long paste: 5 seconds
Count Sent: 275 Count Received: 275
Completed Sequence for ID: 33044
3
Delay for long paste: 1 seconds
Count Sent: 411 Count Received: 411
Completed Sequence for ID: 33050
4
Delay for long paste: 3 seconds
Count Sent: 1358 Count Received: 1358
Completed Sequence for ID: 33072
5
Delay for long paste: 3 seconds
Count Sent: 206 Count Received: 206
Completed Sequence for ID: 33073
6
Delay for long paste: 3 seconds
Count Sent: 309 Count Received: 309
Completed Sequence for ID: 33074
7
Delay for long paste: 1 seconds
Count Sent: 610 Count Received: 610
Completed Sequence for ID: 33076
8
Delay for long paste: 6 seconds
Count Sent: 3248 Count Received: 3248
Completed Sequence for ID: 33081
9
Delay for long paste: 4 seconds
Count Sent: 2124 Count Received: 2124
Completed Sequence for ID: 33084
10
Delay for long paste: 2 seconds
Count Sent: 784 Count Received: 784
Completed Sequence for ID: 33095
11
Delay for long paste: 1 seconds
Count Sent: 568 Count Received: 568
Completed Sequence for ID: 33102
12
Delay for long paste: 2 seconds
Count Sent: 873 Count Received: 873
Completed Sequence for ID: 33108
13
Delay for long paste: 3 seconds
Count Sent: 1521 Count Received: 1521
Completed Sequence for ID: 33127
14
Delay for long paste: 3 seconds
Count Sent: 1404 Count Received: 1404
Completed Sequence for ID: 33128
15
Delay for long paste: 3 seconds
Count Sent: 378 Count Received: 378
Completed Sequence for ID: 33129
16
Delay for long paste: 5 seconds
Count Sent: 2496 Count Received: 2496
Completed Sequence for ID: 33134
17
Delay for long paste: 5 seconds
Count Sent: 139 Count Received: 139
Completed Sequence for ID: 33150
18
Delay for long paste: 5 seconds
Count Sent: 2799 Count Received: 2799
Completed Sequence for ID: 33248
The first report showed a dramatic difference between what was sent and what was received. I suspect an Internet hiccup in that case. The new code to show byte differences failed due to use of a > symbol instead of a < symbol.
That has been corrected in the most recent update, which will run over a small set of posts now...
Delay for long paste: 3 seconds
Text received does not match sent. Skipping to Next Item
Count Sent: 1521 Count Received: 1521
Byte at location: 33 differs: Sent is: 146 Hex:92 and Received is: 63 Hex:3F
Byte at location: 313 differs: Sent is: 146 Hex:92 and Received is: 63 Hex:3F
Byte at location: 329 differs: Sent is: 146 Hex:92 and Received is: 63 Hex:3F
Byte at location: 430 differs: Sent is: 146 Hex:92 and Received is: 63 Hex:3F
Byte at location: 866 differs: Sent is: 146 Hex:92 and Received is: 63 Hex:3F
Byte at location: 1084 differs: Sent is: 147 Hex:93 and Received is: 63 Hex:3FStarting Sequence for ID: 33128
Well! There is is! The high order bit is not passing through the serial line...
I set up the serial line for 8 bits no parity, but somewhere along the line, those instructions are ignored.
The ASCII character with a value of 63 is a question mark.
That's not a major issue for those old posts, but I'm still interested in preparing the Auditor to run in April.
Update at 16:35 local time ... half way through the 18 skips ... taking a break
Update at 19:16 local time ... all 18 posts were manually delivered ... all but one contained special characters that are not part of the standard 7 bit ASCII character set.
Today's Inspector will run shortly. i expect it to finish before 21:30 local time.
Completed Sequence for ID: 33300
Total Command Lines found: 31
Total input Lines in script: 151Number of ID's processed: 300
Starting Number: 33001
Last Number of Run: 33300
Summary for Web Automation Report for 03-21-2022 at 21:12:10
Average time of Loop from Main form: 00:00:22Total time of Processing: 01:49:33
Total time Program was Active: 01:51:37
(th)
Offline
For SpaceNut re Posts Repair initiative ...
Today's runs will try for 33600.
Yesterday's high water mark was 18 skips ... I sure hope our members who used extended ASCII for Apostrophe's and Smiley Faces do not outdo ** that ** record! I suspect the software I am using to move data from one process to another is the cause of the problem. The problem is sufficiently aggravating I might be motivated to do something about it. Inter-process communication is the issue at hand. I spent months looking at a wide range of alternatives before settling on the present system, and (at the time) it did not occur to me that extended ASCII might have been part of the text in the archived posts.
The Trailbreaker reached 33600 with a Green Screen.
Completed Sequence for ID: 33600
Total Command Lines found: 50
Total input Lines in script: 165Number of ID's processed: 300
Starting Number: 33301
Last Number of Run: 33600
Summary for Web Automation Report for 03-22-2022 at 12:45:14
Average time of Loop from Main form: 00:00:42
20 Skip Exceptions were recorded.Total time of Processing: 03:31:26
Total time Program was Active: 04:44:14
20 Skips is a new high water mark. 5*20 >> 100 minutes estimated manual update time
Update at 16:15 local time ... That session lasted over three hours (with plenty of breaks)
The incentive to correct this situation is strong ... The software documentation for the Interprocess Communications Utility says that ** it ** can handle Extended ASCII ...
There are three other points where failure may be occurring...
A serial port is used to send data out... That port is driven by the Microsoft RS232 utility I enlisted to handle serial communication. The place to look for that is the setup of the Microsoft RS232 utility.
The data from the Microsoft utility is turned over to a USB/Serial port device, which itself should not be trimming the high order bit. However, the input port to the Interprocess Communication Utility needs to be checked to be sure it is set for 8 bits no parity.
In the mean time, the Inspector will now review today's 300 posts ...
Update at 19:02 local time ... The Inspector found two potholes, as expected...
What is surprising (to me for sure) is that it also reported a delete ...
That should not have been possible ...
The post of concern is 33383, which I worked on earlier ...
I corrected a couple of posts I'd worked on earlier ... I'll rerun the inspector to be sure everything is cleaned up.
Update at 21:16 local time...
The Inspector came up clean as a whistle this time.
I'm hoping changing the setting of the option in the Interprocess communications utility will make a difference tomorrow. The run then will be a try for 33900.
(th)
Offline
For SpaceNut re Posts Repair initiative...
This is a separate post because it is intended to report a setting change ... After looking at the physical serial connection, and at the Microsoft Utility used to send data through that line, I took at look at the settings for the Interprocess Communications Utility I've been running to move data from WBA into the browser.
To my surprise, I found a setting that allows selection of either:
<> Extended ASCII or
<> Extended ANSI
The default seems to be Extended ANSI. In any case, that is where it was sitting.
I changed it to Extended ASCII, which (by definition) should allow high order bits to be carried.
I'm sure hoping tomorrow's Trailbreaker is able to show improvement.
The manual labor needed to update posts rejected due to character transformations is significant, and there are numerous opportunities for errors.
Meanwhile, the Inspector is going over today's batch a second time, because it found two errors I'd made during the manual update of 18 posts.
(th)
Offline
For SpaceNut re Large Ship project ...
Here is the current list of topics to allow focus on specific aspects of Large Ship:
Large Ship (Prime) Bill of Material / Mass Allocation by tahanson43206
Interplanetary transportation 8 Today 08:19:09 by tahanson43206Sticky: Large scale colonization ship by RobertDyck [ New posts ] [ 1 2 3 … 50 ]
Interplanetary transportation 1,237 Today 01:06:32 by kbd512Sticky: Large Ship Atomic Propulsion by tahanson43206
Interplanetary transportation 11 2022-03-21 16:31:37 by RobertDyckLarge Ship Assembly and Checkout on Orbit by tahanson43206
Interplanetary transportation 7 2022-03-20 17:47:30 by SpaceNutLarge Ship Power Supply Solar Nuclear Other by tahanson43206
Interplanetary transportation 2 2022-03-20 13:48:46 by SpaceNutLarge Ship Component Manufacture on Earth by tahanson43206
Interplanetary transportation 1 2022-03-19 20:59:32 by SpaceNutSticky: Large Ship Cabin Design - Manufacture - Testing - Operation by tahanson43206
Interplanetary transportation 9 2022-03-19 20:51:41 by SpaceNutSticky: Large Ship Physical Fitness Systems by tahanson43206
Interplanetary transportation 3 2022-03-13 19:51:02 by SpaceNutSticky: Large Ship Chemical Propulsion by tahanson43206 [ 1 2 3 ]
Interplanetary transportation 50 2022-03-08 00:59:21 by SpaceNutSticky: Large Ship Radiation Protection by tahanson43206 [ 1 2 3 ]
Interplanetary transportation 51 2022-03-05 10:36:23 by SpaceNutSticky: Large Ship hull material by RobertDyck
Interplanetary transportation 21 2022-03-03 00:08:01 by kbd512Sticky: Large Ship Competing Designs by tahanson43206 [ 1 2 ]
Interplanetary transportation 46 2022-03-01 22:07:52 by SpaceNutSticky: Large Ship Orientation Navigation Pointing by tahanson43206
Interplanetary transportation 9 2022-02-27 10:45:40 by RobertDyckSticky: Large Ship Momentum (Mass) Management by tahanson43206
Interplanetary transportation 8 2022-02-26 10:31:01 by tahanson43206Sticky: Large Ship Food Production Storage Recycling by tahanson43206
Interplanetary transportation 2 2022-02-23 09:11:05 by tahanson43206Sticky: Large Ship Food Preparation and Delivery by tahanson43206
Interplanetary transportation 7 2022-02-22 23:20:22 by SpaceNutSticky: Large Ship Engineering Model by tahanson43206 [ 1 2 ]
Mars Analogue Research Stations 29 2022-02-22 21:56:57 by SpaceNutLarge ship analog by SpaceNut [ 1 2 3 ]
Mars Analogue Research Stations 58 2022-02-19 23:34:34 by SpaceNut
It appears there may be a need for a topic dedicated to construction technique.
The vast majority of construction will be performed on Earth, so that modules that will fit in a Starship can be shipped to LEO for assembly.
The vast majority construction in LEO will be performed by teleoperation by workers on Earth.
If there is a human being on site, that person will be highly paid, given all the care and attention of a NASA astronaut, and will have very little to do, if the teleoperation machinery is well designed.
(th)
Offline
For SpaceNut re Post Repair initiative...
Today's runs will try for 33,900.
The ** big ** test for today is to see if a setting change made in configuration of the Inter-process communications utility will allow Extended-ASCII characters to flow through the serial line without having the high order bit lopped off.
WBA really struggled today ... it halted twice, and there were over 40 skips.
Completed Sequence for ID: 33900
Total Command Lines found: 50
Total input Lines in script: 165Number of ID's processed: 55
Starting Number: 33846
Last Number of Run: 33900
Summary for Web Automation Report for 03-23-2022 at 20:47:25
Average time of Loop from Main form: 00:00:36
5 Skip Exceptions were recorded.Total time of Processing: 00:33:30
Total time Program was Active: 00:44:04
The change from ANSI to Extended ASCII certainly didn't help, as I had hoped it might.
I'll put the setting back to ANSI tomorrow.
Skips report ... the first set of skips were mostly mismatch ... there were a few potholes.
The Inspector will run after i correct the 5 skips in this segment.
The Inspector finished it's run and earned it's pay! Because there were so many skips during the Trail Breaker series of three phases, I missed onel Inspector caught it, so i just fixed itl
If this number of skips continues, I'm going to be motivated to finish updates to a utility program to help deal with skips.
tomorrow's runs will try for 34200 ..
(th)
Offline
For SpaceNut ... this is a special item, not related to Post Repair ...
In the thousands of posts you have made since 2004, here and there are hints that you may have (or had) a chance to interact with the disabled community in some way. There are many folks who provide services to disabled folks, and most go unsung, except for the appreciation they might earn from those they help.
The purpose of **this** post is to open a line of conversation with you about the potential for disabled persons to perform a great variety of useful activities using teleoperation. I am thinking of Large Ship assembly as a starting point, but in fact, there are millions (if not billions) of useful services that might be performed in the "real world" if teleoperation were further advanced.
Today, we have military examples of reliable, effective, efficient teleoperation.
There are (apparently) a ** few ** examples of commercial application of teleoperation. I'm thinking of mining equipment operation in Australia, as an example.
Tomorrow, I would expect teleoperation to grow rapidly to become ubiquitous, and the services provided greatly extended.
(th)
Offline
For SpaceNut re Post Repair initiative...
A Trailbreaker to 34200 is in progress ... I expect it will generate many skips.
A utility program to try to help decrease the time it takes to process skips is in progress as well.
Completed Sequence for ID: 34200
Total Command Lines found: 50
Total input Lines in script: 165Number of ID's processed: 300
Starting Number: 33901
Last Number of Run: 34200
Summary for Web Automation Report for 03-24-2022 at 12:00:52
Average time of Loop from Main form: 00:00:38
15 Skip Exceptions were recorded.Total time of Processing: 03:11:21
Update at 14:44 ... manual edit of 15 posts from today's TrailBreaker is completed....
I'm trying to find a manual procedure that lends itself to automation ...
The Inspector will start now. I don't expect any defects to be reported.
Update at 17:25 local time... the Inspector caught two posts I missed.
We should be clear through 32,400 at this point.
(th)
Offline
For SpaceNut re Posts Repair initiative ... a Trailbreaker to 34500 just finished
Completed Sequence for ID: 34500
Total Command Lines found: 51
Total input Lines in script: 167Number of ID's processed: 300
Starting Number: 34201
Last Number of Run: 34500
Summary for Web Automation Report for 03-25-2022 at 11:51:57
Average time of Loop from Main form: 00:00:38
8 Skip Exceptions were recorded.Total time of Processing: 03:11:52
8 skips is a lot better than 40+
Update at 13:03 local time ... the eight manual updates were completed.
This is a painstaking process, which I would automate if I could, but at present, I don't have a solution for the problem of inter-process communication altering characters. If anyone who chances to read this post has a clue ab out how to transfer data between two independent processes without altering bits, i'd be most interested.
The process I am using is courtesy of folks who developed a tool for handicapped assistance. They allow developers of assistance hardware to enable operators to work with browsers without a keyboard or mouse. The software ** claims ** to transfer bytes of the Extended ASCII character set, but clearly there is a function in the flow from program to browser that causes high order bits to be lost.
SpaceNut ... for the record, here is the procedure:
Starting with output from a Trailbreaker run:
1) Make a work copy
2) Using work copy, perform replace so that an edit url takes the place of log text before ID that failed
3) Using search on work copy, find each defective ID in turn
4) At each defective ID, begin by copying the URL for edit
5) Clear the active highlighted state of the text in the work document
6) Alt-Tab to reach the browser
7) Ctrl-l to access the address bar
8) Ctrl-v to deliver the URL to the browser
9) Enter to cause the browser to call up the defective post
10) Return to work document using Alt-Tab
11) On work document, highlight the output of the Edit process (post without bandits)
12) Ctrl-A and then release the highlighted text
13) Alt-Tab to the browser ... the cursor should be in the Edit window
14) Ctrl-A and Ctrl-V to deliver the edit output to the Edit window
15) Using mouse, click Submit
16) Visually verify that the delivered post looks like the edit output
17) Alt-Tab back to the work document
Repeat until all defective posts have been processed.
I'll now start the Inspector, to double check the work.
Update at 16:02 local time ... The Inspector reported a clear series.
We appear to be good through 34,500.
Tomorrow's runs will try for 34,800.
(th)
Offline
For SpaceNut re Posts Repair and other ongoing activities ...
Reminder: The ID renovation is holding at 11199 ... there are several thousand ID's to upgrade from banned to Member status.
The work of finishing those can be automated ... kbd512 demonstrated a method, but stopped work for some reason.
I'm hoping to make progress in preparing the Auditor script (and the data to feed it) to correct posts that had special characters changed to question marks.
Meanwhile, the daily runs will try for 34800 today.
newmars.com/forums/edit.php?id=34566 Skipping to Next Item
done
Completed Sequence for ID: 34612 pothole
done
newmars.com/forums/edit.php?id=34675 Skipping to Next Item
done
newmars.com/forums/edit.php?id=34700 Skipping to Next Item
done
newmars.com/forums/edit.php?id=34719 Skipping to Next Item
doneCompleted Sequence for ID: 34800
Total Command Lines found: 51
Total input Lines in script: 167Number of ID's processed: 300
Starting Number: 34501
Last Number of Run: 34800
Summary for Web Automation Report for 03-26-2022 at 12:49:23
Average time of Loop from Main form: 00:00:38
6 Skip Exceptions were recorded.
Update at 13:23 local time .. that's interesting (to me at least) ... The log says there were 6 skips but I only four five. The Inspector will find the missing item ... it is not a pothole, which is easy to find: "Bad request"
Update at 15:30 local time ... The Inspector just finished with a Green screen ...
I'm hoping it found the skip I missed during/after the Trailbreaker run ...
Completed Sequence for ID: 34800
Total Command Lines found: 31
Total input Lines in script: 151Number of ID's processed: 300
Starting Number: 34501
Last Number of Run: 34800
Summary for Web Automation Report for 03-26-2022 at 15:31:12
Average time of Loop from Main form: 00:00:22
1 Skip Exceptions were recorded.Total time of Processing: 01:48:59
That skip would have been the pothole reported earlier. Now I'll check for the missing skip...
Found it! 34670 !!!
Ah yes! [From: Nova Scotia, Canada]
This gent was in the habit of coding "unquote" instead of the format preferred by FluxBB.
The post is corrected and looking spiffy!
We appear to be good through 34800.
Tomorrow's runs will try for 35100.
Preparations for an Auditor run against the first 30,000 posts made inchworm progress today.
Assembly of a unified input file for the run is complicated by a re-run from 1-18000, due to discovery of skipping of posts that were not reported. That April 1st due date is looking iffy, at the present (slow) rate of progress.
A unified file would contain all the Edit outputs for every post, and the Edit outputs are scattered between the two sets of runs.
(th)
Offline
For SpaceNut re new BOM topic ....
This topic is not developing as I had hoped....
There should be NO discussion in the topic.
All discussion should be happening in other topics.
The BOM topic is intended to be the final result posting site.
There should be NO further discussion of hull plate panels, except to finalize the remaining details for a specification.
Hopefully we'll wrap that up at the next Zoom meeting.
If we were designing a ship for use on Earth on water, we might start with the keel, if the ship is large enough to have one.
Smaller craft (apparently) do not have keels, but instead are defined by hull shapes.
In the case of Large Ship, there is no keel to lay.
The equivalent of the keel is where I have decided to start fabrication. The Habitat Ring is the distinctive feature of the ship. The hull plating decision frees the design committee to proceed to the next critical component set, which is the set of "formers" and "stringers" to which the hull plates are to be welded.
Everything in Large Ship will flow naturally and (hopefully) effortlessly from this foundation element.
There should be no need for concern about unknown unknowns.
Everything in Large Ship ** should ** flow naturally and easily from the hull plate decision, just as everything in a "real" ship flows naturally and easily from a keel.
(th)
Offline
For SpaceNut re Post Repair initiative ...
Today's Trailbreaker finished with a Green screen.
Completed Sequence for ID: 35100
Total Command Lines found: 51
Total input Lines in script: 167Number of ID's processed: 300
Starting Number: 34801
Last Number of Run: 35100
Summary for Web Automation Report for 03-27-2022 at 13:09:56
Average time of Loop from Main form: 00:00:39
25 Skip Exceptions were recorded.Total time of Processing: 03:16:54
Good grief! 25 Skips!
Update at 14:23 ... this set of 25 updates took just over an hour ... performance per items is improving a bit
The Inspector will review the work now ...
Follow up: The Inspector found no errors.
(th)
Offline
For SpaceNut re Post Repair initiative ...
Today's Trailbreaker finished with a Green screen ...
Completed Sequence for ID: 35400
Total Command Lines found: 51
Total input Lines in script: 167Number of ID's processed: 300
Starting Number: 35101
Last Number of Run: 35400
Summary for Web Automation Report for 03-28-2022 at 14:02:49
Average time of Loop from Main form: 00:00:39
21 Skip Exceptions were recorded.Total time of Processing: 03:14:26
21 skips ... that'll take a solid hour if I'm lucky
***
In other news .. I've decided to take another tack in approaching the Auditor script.
To review, the first 30,000 posts were processed with a script that did not compare the text submitted to the browser to the text output from the Edit function. That problem has been corrected, but the first 30,000 posts include some number whose special (Extended ASCII characters) were converted to question marks.
Preparing the log files for automation of repair is going to take far longer than the days left in March, and i want to perform the Auditor runs in April, so that 1000 posts can be audited per day for 30 days.
The solution I'm proposing is a modified script to find question marks in the active posts.
A few posts contain question marks, but there are few of those, because few posters ask questions. The vast majority appear to be offering statements of various kinds.
What is clear is that whatever mechanism was at work damaging posts was substituting question marks for characters with high order bit set to 1.
An audit designed to count question marks in posts will provide a way to identify posts that need attention.
In addition, the audit (as designed) will provide a count of characters in the live posts.
This count can be compared to the Edit output in the log files, and the occasional lost data posts can be identified and corrected.
Use of this read-only script will allow for collection of needed error data without requiring edit mode.
Update at 14:22 ... The new Auditor script needs a test run ... I've decided to look at 30,001-31000.
Those posts ** should ** be free of errors, but there is no harm in looking at them.
Update at 16:49 local time after trial run of Auditor script ... that was disappointing ... 161 out of 336 posts contained question marks put their by the authors.
My hopes for a quick and easy shortcut to find damaged posts are reduced significantly.
Update at 20:53 local time ... 21 manual edits took 42 minutes ... a definite improvement.
The Inspector will now double check the work.
Update at 23:04 local time ...
The Inspector just finished with a Green screen ... I'll check for skips or deletes... None.
A clean run ... Tomorrows runs will try for 35700
(th)
Offline
For SpaceNut re Post Repair Initiative ...
The Trailbreaker for today is under way... the target is 35700.
After comparing the results from the Auditor script and the Inspector, I have decided to go with Inspector for the April run over 1-30000. Inspector delivers the contents of the FluxBB stored post, which I can compare to the original post stored in earlier logs. The time crunch is removed by taking this step.
It is unlikely the Inspector will find any posts that have bandits, since that stretch of posts has been covered twice, but it ** will ** collect the current state of the posts, which is helpful.
The problem of trying to find the original posts in all the logs from December 1st of 2021 through the end of the 30,000 sequence is proving more challenging than I had imagined, primarily because the scripts and WBA itself were improved along the way.
It would have been ** really ** helpful if I had thought of comparing the data delivered to the browser to the data output from the Edit process, but I trusted that a data flow within the PC would be reliable, and (of course) I was wrong.
The combination of script and program running now **are** catching any dropped bits at this point, but that's no consolation for all those thousands of posts from the periord before 2004 that were used for learning the job.
Update at 12:19 ... Trailbreaker finished with a Green screen ...
Completed Sequence for ID: 35700
Total Command Lines found: 51
Total input Lines in script: 167Number of ID's processed: 300
Starting Number: 35401
Last Number of Run: 35700
Summary for Web Automation Report for 03-29-2022 at 12:17:28
Average time of Loop from Main form: 00:00:38
17 Skip Exceptions were recorded.Total time of Processing: 03:09:59
Total time Program was Active: 03:10:46
17 skips .... That should take under an hour
The updates took 30 minutes of online time. That does not include preparation time.
Preparation involves collecting a list of the posts to receive attention, and then pasting the fetch URL ahead of each of them.
The Inspector ** should ** find all those posts are clean as a whistle!
The inspector found one pothole (as expected) and no deletes, also as expected.
Tomorrow's runs will try for 36000.
A folder is prepared to hold the planned Inspector runs that will begin April 1 and run for 30 days, to capture the current contents of the MySQL database for those posts.
At a later time, those posts will (if time permits) be compared to the originals output from the Edit function during the original update series.
(th)
Offline
For SpaceNut re Large Ship project structure ....
This is a list of topics supporting Large Ship as of Tuesday 2022/03/29
Large scale colonization ship by RobertDyck [ 1 2 3 … 50 ]
Interplanetary transportation 1,242 Yesterday 20:15:39 by RobertDyck
Large Ship Calculations by tahanson43206
Interplanetary transportation 5 Yesterday 23:02:58 by tahanson43206
Large Ship (Prime) Bill of Material / Mass Allocation by tahanson43206 [ 1 2 ]
Interplanetary transportation 27 Yesterday 21:27:03 by SpaceNut
Large Ship Component Manufacture on Earth by tahanson43206
Interplanetary transportation 10 Yesterday 21:23:45 by SpaceNut
Large Ship Assembly and Checkout on Orbit by tahanson43206
Interplanetary transportation 9 Yesterday 21:17:37 by SpaceNut
Large Ship Cabin Design - Manufacture - Testing - Operation by tahanson43206
Interplanetary transportation 10 2022-03-27 18:43:58 by SpaceNut
Large Ship Radiation Protection by tahanson43206 [ 1 2 3 ]
Interplanetary transportation 53 2022-03-26 14:26:45 by SpaceNut
Large Ship Power Supply Solar Nuclear Other by tahanson43206
Interplanetary transportation 2 2022-03-20 13:48:46 by SpaceNut
Large Ship Physical Fitness Systems by tahanson43206
Interplanetary transportation 3 2022-03-13 19:51:02 by SpaceNut
Large Ship Chemical Propulsion by tahanson43206 [ 1 2 3 ]
Interplanetary transportation 50 2022-03-08 00:59:21 by SpaceNut
Large Ship Orientation Navigation Pointing by tahanson43206
Interplanetary transportation 9 2022-02-27 10:45:40 by RobertDyck
Large Ship Momentum (Mass) Management by tahanson43206
Interplanetary transportation 8 2022-02-26 10:31:01 by tahanson43206
Large Ship Food Production Storage Recycling by tahanson43206
Interplanetary transportation 2 2022-02-23 09:11:05 by tahanson43206
Large Ship Food Preparation and Delivery by tahanson43206
Interplanetary transportation 7 2022-02-22 23:20:22 by SpaceNut
Large Ship Engineering Model by tahanson43206 [ 1 2 ]
Mars Analogue Research Stations 29 2022-02-22 21:56:57 by SpaceNut
Large ship analog by SpaceNut [ 1 2 3 ]
Mars Analogue Research Stations 58 2022-02-19 23:34:34 by SpaceNut
Large Ship hull material by RobertDyck [ New posts ] [ 1 2 ]
Interplanetary transportation 28 Today 06:49:54 by RobertDyck
Large Ship Competing Designs by tahanson43206 [ 1 2 ]
Interplanetary transportation 46 2022-03-01 22:07:52 by SpaceNut
Large Ship Atomic Propulsion by tahanson43206
Interplanetary transportation 12 2022-03-25 21:01:06 by Calliban
***
I'm planning to add a Database topic today.
(th)
Offline
Not sure what the data base should have in it other than the solid attributes for a build.
Starting with the image, dimensions of legnth/ witdth/ height; volumes and area numbers.
Then working to the mass for items such as the ring, tunnels and hub center followed by air with its life support such as the greenhouse area and such.
Offline
For SpaceNut re #2093
Thank you for the interesting "question"
In saying that it is not clear what a database would have in it, you are asking a question that surely many others might be asking.
In starting to think about the Large Ship database, my first concern is that whatever data resides there should be easy to find when it is needed.
That is (of course) the exact opposite of a longitudinal Forum structure such as ours. Of course you know that the forum runs as a database application.
Data of various kinds is stored in tables which have indexes, and software is able to collect data as it is needed rapidly.
Almost everyone who is a regular user of the forum software is using indexed tables. It is the beauty of well written software, and an efficient database, that performance is fast, and results are (generally) satisfactory. It is disappointing when a search turns up nothing, but that is not the fault of the database.
In the case of Large Ship, ultimately there will be thousands of parts to keep track of, from manufacture through the last day of service.
In addition, there will be hundreds of vendors to keep track of, and (probably) thousands of contracts for products or services or both.
A good example of a database application is tracking the 119 panels that we (Zoom committee) has decided to select as the elements that will together make up a seamless sheet of metal to hold air inside the Habitat Ring.
This is an example of a 1 to many relationship, which Relational Databases are designed to support.
There is 1 ship, and (I expect) it will have a serial number of 1.
However, in the years ahead, there will be many more identical ships, or ships of the same class, so Serial Number 1 will be followed by others.
Serial Number 1 will have a Habitat Ring, and the Habitat Ring will have 119 Hull panels.
Each Hull Panel will have a minimum of 1 stringer and a minimum of 2 former elements.
The welds to hold the formers and the stringer to the panel must be documented as tasks assigned to a person (teleoperator), completed and inspected.
The individual pieces will be the responsibility of a specific individual, who will follow the progress of the panel from procurement of components, through manufacture and shipment, to final assembly and inspection at the assembly station.
A relational database can handle all those data tracking requirements, and millions of others, efficiently and reliably, assuming the humans who interact with it provide accurate information.
(th)
Offline
For SpaceNut re Posts Repair initiative...
Today's Trailbreaker just ended with a Green screen.
To my surprise (and relief) ONLY 8 skips were reported.
Completed Sequence for ID: 36000
Total Command Lines found: 51
Total input Lines in script: 167Number of ID's processed: 300
Starting Number: 35701
Last Number of Run: 36000
Summary for Web Automation Report for 03-30-2022 at 12:54:47
Average time of Loop from Main form: 00:00:39
8 Skip Exceptions were recorded.Total time of Processing: 03:18:02
Total time Program was Active: 03:55:12
Skip report to follow
All 8 skips were handled in 36 minutes, from start to finish, for an average of just over 4 minutes per.
That average time is likely to improve as the days go by, because the mechanical procedures seem to be memorizable, and the sequence of actions seems to be coming more quickly.
There are key check points to verify that work is progressing properly.
The Inspector will now check the work.
Clean run!
Completed Sequence for ID: 36000
Total Command Lines found: 31
Total input Lines in script: 151Number of ID's processed: 300
Starting Number: 35701
Last Number of Run: 36000
Summary for Web Automation Report for 03-30-2022 at 17:10:32
Average time of Loop from Main form: 00:00:22Total time of Processing: 01:49:13
No skips! Delete count: zero
(th)
Offline
For SpaceNut re posts repair initiative ...
Today's Trailbreaker finished with a Green screen..
Completed Sequence for ID: 36300
Total Command Lines found: 51
Total input Lines in script: 167Number of ID's processed: 300
Starting Number: 36001
Last Number of Run: 36300
Summary for Web Automation Report for 03-31-2022 at 13:05:42
Average time of Loop from Main form: 00:00:38
14 Skip Exceptions were recorded.Total time of Processing: 03:08:28
14 skips ... it could have been more - that's a good hour of manual procedure ...
All 14 are taken care of, at the rate of 3 minutes per item including one break.
There were three potholes and the rest were mismatches ... there were no rejects.
We may be past our member who liked "unquote" instead of the preferred form.
The Inspector will start shortly ... it will find the three potholes, of course, but hopefully nothing else.
***
Looking ahead ... April starts tomorrow. i'll begin full scale Inspector runs over 1000 posts, after daily runs.
The goal for April is to collect a complete set of the existing posts on FluxBB's MySQL database.
We accumulated a similar set of posts back when we ran for first scan for bandits.
There are two sets of updates on file. The first was performed over the period from December of 2021.
The second was a set of redo's of the first 18,000 posts.
The dataprocessing challenge is to find all the Edit outputs for all those posts, while also noting potholes.
Fortunately, all that work can be performed off line.
When ** that ** work is complete, the output ** should ** be an accurate list of posts that need attention.
I am hoping the number is small, but current experience suggests it may amount to:
14 per 300 current .... 30000 total so 100 batches of 300 so 1400 posts may need attention.
I'm not worried about the ones that lost their Smiley faces. The ones I'm worried about are the ones that lost content because the Internet was choppy and we weren't checking sent vs received, as we are now.
Hopefully there aren't a lot of those. The offline processing should find them.
Update at 17:05 local time:
As expected, Inspector found four potholes and no other issues.
Tomorrow's runs will include:
1) Trailbreaker: 36600
2) Inspector: 36600
3) Inspector: 1-1000
(th)
Offline
For SpaceNut re NewMars outage....
FYI ... NewMars was down at 2 AM UTC... I found it alive and well at 11:30 UTC
For the record, here is a Ping record taken at 11:30 UTC:
PING newmars.com (69.163.178.194) 56(84) bytes of data.
64 bytes from ps583973.dreamhostps.com (69.163.178.194): icmp_seq=1 ttl=47 time=80.3 ms
(th)
Offline
For SpaceNut re Posts Repair initiative ...
Today's runs will try for 36600.
Trailbreaker is in motion.
As a side note ... today's NewMarsMember inbox contained an advertisement from DropBox ... I saved it in case there is ever a need for the service...
Apparently they are offering a verified correspondence service of some kind. it reminds me of a business practice I've seen, in which a contract is maintained by a server, and both/all participants make commitments electronically.
DropBox management may have seen a business opportunity because of their standing in the marketplace.
Update noonish local time ... Trailbreaker finished with a Green screen and only 12 skips
I call it a good day when skips are fewer than 20!
Still, it will be most of an hour to take care of them manually.
I'm open to input from anyone who knows a technique for inter-process communication that delivers ** all ** data.
In the mean time, it's off to the manual clean up "hurdles". The "hurdle" metaphor is a lot closer to the experience than a simple race. Or maybe a tennis match where the ball ** has ** to end up on the far side of the net is even better as a metaphor. The manual process involves multiple shifts back and forth between the log (in Notepad) and the Edit page (in the browser).
Just for the fun of it, I decided to show you (SpaceNut) what the intermediate output looks like:
Text received does not match sent for ID: 36317 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36317
Text received does not match sent for ID: 36323 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36323
Text received does not match sent for ID: 36357 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36357
Text received does not match sent for ID: 36360 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36360
Text received does not match sent for ID: 36362 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36362
Text received does not match sent for ID: 36376 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36376
Text received does not match sent for ID: 36379 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36379
Text received does not match sent for ID: 36393 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36393
Text received does not match sent for ID: 36418 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36418
Text received does not match sent for ID: 36420 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36420
Text received does not match sent for ID: 36424 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36424
Text received does not match sent for ID: 36465 Skipping to Next Item
Time Out! Skip Requested! Skipping to Next from Post: 36465
This particular set consists entirely of Smileys and European Apostrophes.
On other days, we can see a variety of inventions by our former members, that FluxBB rejects.
Our friend from "unquote" land is not present today.
Update at 13:50 local time ... the updates are completed.
The Inspector will now go over today's 300 posts, to confirm the work was done correctly.
Update at 15:44 local time ... the Inspector finished with a Green screen.
There were no skips, as expected, since there were no potholes in the Trailbreaker.
Now checking for Deletes ...Primo! None! We have 300 clean as a whistle.
This is April 1, so the new re-scan of the first 30,000 posts will start now.
A run over 1000 posts will take 22*1000 >> 22000 / 3600 >> 6 hours (or so) so ETA 20:00 (or so)
Since the script is the Inspector, it will report potholes as skips, and it will report any posts missed with deletes.
The main concern I have for this re-scan is finding short posts (lost data) and lost Smiley's.
Fortunately, all original data was preserved in the first scan to 77,000, so while it may take a few months to clean everything up, there is no reason (other than volunteer time) it can't be achieved.
In the mean time, all new work (since 30,000) is proceeding with bit-by-bit checking in place.
Update at midnight local time;
The first Inspector run finished ... it took 8 hours to cover 1000 posts.
It reported 19 skips, which would be potholes. More importantly, it found NO deletes, so the update runs caught all the bandits. On the other hand, the only way to be ** sure ** we have a clean set of posts is to compare the ones collected in this run, with the outputs of the Edits. That will happen (hopefully) some time in the future.
The laptop will not get a lot of breaks over the next month...
The regular Trailbreaker takes three hours.
The inspector takes two hours.
This new full 1000 Inspector takes 8 hours.
I'll give the laptop a break now.
(th)
Offline
For SpaceNut ... this post is about the 1950 census, which was just released.
https://www.archives.gov/developer/1950 … setbystate
There are a lot of options. I'd be interested in seeing the census forms filled out by relatives where were alive and raising families at the time.
I did find a particular city in a particular state, but didn't see what to do after that.
I'd be interested in your assessment of the data set, and how to search for particular people.
(th)
Offline
Rather than an actual page it appears to get the data you are creating a cmd line string in the browser address bar.
Offline