Hello All!

So, You may have noticed there are some messages in the FSX_STA echo showing some stats for Hub 4. I had been working on an MPL that would generate those stats files, but ran into issues when trying to automate the running of Mystic via a cronjob.

What I ended up doing, was rewriting it in FreePascal, and have been testing it locally here for a couple days. It seems to be working just fine, but we all know how that goes... ;)

The one posted last night, almost 12 hours ago, will give a general idea of the information it will contain. I have made a few changes to it, such as adding the time of the nodes last poll.

This program scans all of the fileboxes, and counts up the total files and size of what's waiting for each node, and then spits out a report. On the bottom of this report, there is a list of nodes that haven't polled for x number of days. It also gives a brief status of where that node is at in the removal process. For example, node 21:4/154 is now showing as not connecting for 89 days, and the status is 'Removal Warning!'. Once that hits 90 days, the status will change to 'Remove Node!!!'. This tells me that it is now time to remove this node, and remove all files from their filebox.

Just a quick and dirty tool to make sure things continue to run smoothly over here, and help me to not have to manually check all of the fileboxes to see what's waiting for each node. ;)


Black Panther

... Eat the rich, then take all their money- Darkwood
--- GoldED+/LNX 1.1.5-b20180707
* Origin: Exploring other frontiers! (21:1/186)

On 24 Oct 2021 at 11:33a, Black Panther pondered and said...

Hello All!

Hey stranger. :)

So, You may have noticed there are some messages in the FSX_STA echo showing some stats for Hub 4. I had been working on an MPL that would generate those stats files, but ran into issues when trying to automate the running of Mystic via a cronjob.

I have seen them, wondering how you were getting on.

What I ended up doing, was rewriting it in FreePascal, and have been testing it locally here for a couple days. It seems to be working just fine, but we all know how that goes... ;)

So far so good I'd say :)

The one posted last night, almost 12 hours ago, will give a general idea of the information it will contain. I have made a few changes to it,
such as adding the time of the nodes last poll.

Look forward to seeing the next iteration soon.

This program scans all of the fileboxes, and counts up the total files
and size of what's waiting for each node, and then spits out a report.

Yes Deon and I are using scripts that shipped with the Husky project (I think that's where they came from?) to do something similar. I find it a bit of a pain as I need to create symbolic links for fileboxes I create in order to accommodate the code I'm running. It wants to see fileboxes in a certain way only. A bit like my passion for rum and raisin ice cream - forget that strawberry stuff :)

On the bottom of this report, there is a list of nodes that haven't
polled for x number of days. It also gives a brief status of where that node is at in the removal process. For example, node 21:4/154 is now showing as not connecting for 89 days, and the status is 'Removal Warning!'. Once that hits 90 days, the status will change to 'Remove Node!!!'. This tells me that it is now time to remove this node, and remove all files from their filebox.

I'm using the reports I can generate for the same reasons as is Deon. I'm not sure I'd start giving warnings at 89 days, probably (ideally) it would be at 35 and remove around 45 days, that's six weeks.

That said, I can be slack and let some nodes drift out much longer or nodes can get in touch and ask to not be removed while they work on getting something broken fixed again etc...

In an ideal world we'd all agree some key numbers for warning, final warning, removal ...

Thoughts anyone?

--- Mystic BBS v1.12 A47 2021/09/29 (Linux/64)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Hello Avon!

25 Oct 21 15:40, you wrote to me:

On 24 Oct 2021 at 11:33a, Black Panther pondered and said...

Hello All!

Hey stranger. :)

They don't get too much stranger than me. ;)

Yes Deon and I are using scripts that shipped with the Husky project
(I think that's where they came from?) to do something similar. I find
it a bit of a pain as I need to create symbolic links for fileboxes I create in order to accommodate the code I'm running. It wants to see fileboxes in a certain way only. A bit like my passion for rum and
raisin ice cream - forget that strawberry stuff :)

I was going to use that one, but I have everything, including the echomail, being placed into fileboxes. That script didn't work with 'filebox always' set. So, I had to come up with my own iteration of it. ;)

At the time I was setting up husky, I thought it would be good to be able to see everything in the nodes directory, instead of all the echomail sitting in the same outbound. It made it difficult to see what belonged to which node.

I'm using the reports I can generate for the same reasons as is Deon.
I'm not sure I'd start giving warnings at 89 days, probably (ideally)
it would be at 35 and remove around 45 days, that's six weeks.

That's just a number I was using for testing purposes. The way this one is set up right now, is start warning at 30 days, next warning at 60, and removal at 90. Once I know everything is working correctly, I'll adjust those numbers.

That said, I can be slack and let some nodes drift out much longer or nodes can get in touch and ask to not be removed while they work on getting something broken fixed again etc...

Agreed. I had thought about automating the removal process within this program, but figured it would be better to do it manually, in those cases where a node has indicated they would be down for a period of time.

In an ideal world we'd all agree some key numbers for warning, final warning, removal ...

I'm thinking 30 days for email notification, another attempt made at 45, and removal at 60. Again, if there is notification given, those dates would be flexible.

Thoughts anyone?

Many. But they all lead to certain death... ;)

--- Mystic BBS v1.12 A47 2021/09/29 (Linux/64)

I'm also still trying to wrap my head around you running on Linux. :)


Black Panther

... Vulcan aerobics - Live long and perspire.
--- GoldED+/LNX 1.1.5-b20180707
* Origin: Exploring other frontiers! (21:1/186)

Black wrote (2021-10-24):

The one posted last night, almost 12 hours ago, will give a general idea of the information it will contain. I have made a few changes to it, such as adding the time of the nodes last poll.

I hope you do understand that these stats are (EU) GDPR* violations, if nodes haven't agreed that you publish stats about their polling behaviour (not sure how Canadian and NZ privacy law sees it).

---
* Origin: 1995| Invention of the Cookie. The End. (21:3/102)

Hello Oli!

25 Oct 21 06:53, you wrote to me:

I hope you do understand that these stats are (EU) GDPR* violations,
if nodes haven't agreed that you publish stats about their polling behaviour (not sure how Canadian and NZ privacy law sees it).

Nope. I'm posting stats about files sitting on my personal system. If I wanted to post information about polling behaviour, I'd post the 9+ MB daily binkd log file.

Black Panther

... Virginity: like a balloon: one prick and it's gone.
--- GoldED+/LNX 1.1.5-b20180707
* Origin: Exploring other frontiers! (21:1/186)

BY: Oli(21:3/102)

I hope you do understand that these stats are (EU) GDPR* violations, if nodes haven't agreed that you publish stats about their polling
behaviour (not sure how Canadian and NZ privacy law sees it).

I think people are more worried about Facebook and Twitter than FTN or QWKnets.


--- WWIV 5.5.1.3261
* Origin: inland utopia * socal usa * iutopia.mooo.com:2023 (21:4/108)

Black wrote (2021-10-25):

Hello Oli!

25 Oct 21 06:53, you wrote to me:

I hope you do understand that these stats are (EU) GDPR* violations,
if nodes haven't agreed that you publish stats about their polling
behaviour (not sure how Canadian and NZ privacy law sees it).

Nope. I'm posting stats about files sitting on my personal system. If I wanted to post information about polling behaviour, I'd post the 9+ MB daily binkd log file.

Ignorance is bliss.

---
* Origin: 1995| Invention of the Cookie. The End. (21:3/102)

Utopian wrote (2021-10-25):

BY: Oli(21:3/102)

I hope you do understand that these stats are (EU) GDPR* violations, if
nodes haven't agreed that you publish stats about their polling
behaviour (not sure how Canadian and NZ privacy law sees it).

I think people are more worried about Facebook and Twitter than FTN or QWKnets.

Facebook or Twitter are irrelevant here.

---
* Origin: 1995| Invention of the Cookie. The End. (21:3/102)