I'm trying to compile some software from source and hitting errors. It seems like it's missing files and I think some of the issue was the compile files were looking for lower case named files but the zip of the source code
had files saved in upper case. I changed the files to lowercase then ran this best guess.

Note, I've not used gcc much at all.

gcc config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp socket.hpp sock.h spoon.cpp spoon.h -o spoon.test

The docs say the software is meant to compile using gcc for Linux, you can
see the files have .cpp extensions as well as .h and .hpp

The docs say there should be a suggested make shell script for gcc with it,
but this seems to be missing in the zip.

The other associated programs have the shell scripts but why the one bit of software I am trying to build is missing it, is anyones guess.

I attempted my above command line based on some other examples where there
was info on how to call gcc... that said I don't know which files to include
in the build and which to leave out.

When I ran this I got the following

In file included from spoon.h:115,
from config.cpp:38:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
In file included from spoon.h:115,
from keyboard.cpp:38:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
In file included from spoon.h:115,
from nntp.cpp:38:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
In file included from spoon.h:115,
from pop.cpp:38:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
In file included from spoon.h:115,
from smtp.cpp:38:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
In file included from socket.cpp:8:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
In file included from spoon.h:115,
from spoon.cpp:40:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.
In file included from spoon.h:115:
socket.hpp:51:8: error: expected identifier or \u2018(\u2019 before string constant
extern "C" {
^~~
In file included from spoon.h:115:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>
^~~~~~~~~~~~
compilation terminated.

iostream.h does not appear to be in the zip file with this source code, could it be some other known file I can source elsewhere?

If there's anyone about that has some experience in these matters or could
help getting this complied, please sing out.

Thanks :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

gcc config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp
socket.hpp
sock.h spoon.cpp spoon.h -o spoon.test

Given they are c++ files, not plain c, you might have better luck with g++

g++ is the C++ compiler, gcc is C compiler.

Andrew


--- Talisman v0.10-dev (Linux/armv7l)
* Origin: HappyLand v2.0 - telnet://happyland.zapto.org:11892/ (21:1/182)

Avon wrote (2021-01-15):

I'm trying to compile some software from source and hitting errors. It seems like it's missing files and I think some of the issue was the
compile files were looking for lower case named files but the zip of the source code had files saved in upper case. I changed the files to
lowercase then ran this best guess.

Note, I've not used gcc much at all.

gcc config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp
socket.hpp sock.h spoon.cpp spoon.h -o spoon.test

soupgate?

I succesfully compiled it on Linux some years ago. Does it work with g++ as apam mentioned. If not I have it zipped as a backup and can look if I made any changes to get it compile. I might discovered some bug, I'm not sure anyomre if it was soupgate or some other software I used with soupgate or just the pktview tool.

---
* Origin: this message must NOT be gated to Telegram (21:3/102)

On 15 Jan 2021 at 12:57p, Oli pondered and said...

soupgate?

I succesfully compiled it on Linux some years ago. Does it work with g++ as apam mentioned. If not I have it zipped as a backup and can look if I made any changes to get it compile. I might discovered some bug, I'm not sure anyomre if it was soupgate or some other software I used with soupgate or just the pktview tool.

Soup, The version that's avail from the website is not the latest it seems
and missing some authenticated login to NNTP that I'm after :(
Thanks for the info and offer :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 15 Jan 2021 at 08:03p, apam pondered and said...

Given they are c++ files, not plain c, you might have better luck with
g++
g++ is the C++ compiler, gcc is C compiler.

Sadly nope, still missing some files called in one of the other files I think.

Oli has provided me with a possible lead so just chasing that up now :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-16):

On 15 Jan 2021 at 12:57p, Oli pondered and said...

soupgate?

I succesfully compiled it on Linux some years ago. Does it work
with g++ as apam mentioned. If not I have it zipped as a backup and
can look if I made any changes to get it compile. I might
discovered some bug, I'm not sure anyomre if it was soupgate or
some other software I used with soupgate or just the pktview tool.

Soup, The version that's avail from the website is not the latest it seems and missing some authenticated login to NNTP that I'm after :(
Thanks for the info and offer :)

So spoon would be the replacement for VSoup? It can feed/create soup packets to/from nntp and smtp/pop3 servers?

Is there a NNTP server that serves these gated messages (from FSXnet?) and can I (anyone) access it? I'm interested how well the gating works.

---
* Origin: this message must NOT be gated to Telegram (21:3/102)

On 16 Jan 2021 at 11:48a, Oli pondered and said...

So spoon would be the replacement for VSoup? It can feed/create soup packets to/from nntp and smtp/pop3 servers?

Yes that's my understanding although I have yet to test all of this.

Is there a NNTP server that serves these gated messages (from FSXnet?)
and can I (anyone) access it? I'm interested how well the gating works.

news.bbs.nz is the server I run that the gateway software at 1/10 connects with. The NNTP server does not (at this time) carry the fsxNet FTN echos.
You should be able to poll it to obtain the usual big 8 Usenet
newsgroups / hierarchies as read access is open to all. Post access requires authenticated login. There are a few groups like nz.test and agency.test you can use though to test against.

Feel free to netmail me with any other questions :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 15 Jan 2021 at 10:51p, Avon pondered and said...

gcc config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp socket.hpp sock.h spoon.cpp spoon.h -o spoon.test

Generally speaking, you don't try to compile header files
(.hpp or .h) directly. Rather, those are usually meant to
be "included" in a source file that is then compiled.
Header files (sometimes called "include files") usually
contain things like structure and type definitions and
function declarations that you want visible in more than
one source file.

Running `gcc` by itself may well be just fine, as it's
generally smart enough to understand what language it is
compiling based on the filename (specifically, the
"extension", though one should be careful to note that
Unix-like systems don't assign any special meaning to
file extensions itself, unlike DOS/Windows/DEC systems;
it leaves assigning meaning to parts of a filename to
user-level programs...I digress). But running g++ or
c++ is also fine and certainly won't hurt anything.

The docs say there should be a suggested make shell script for gcc with it, but this seems to be missing in the zip.

There is no `makefile` or `Makefile`? That's unfortunate.

In file included from spoon.h:115,
from config.cpp:38:
socket.hpp:73:10: fatal error: iostream.h: No such file or directory
#include <iostream.h>

This is one of two things:

1) Since the compiler was invoked as "gcc" and the ".h" file
extension is usually associated with C language header files,
it is trying to compile that file in C mode. However, as
mentioned above, one doesn't generally compiler header
directly; simply omit the ".h" and ".hpp" files from the command
line and see if that helps?

2) <iostream.h> has itself been deprecated for almost 25 years
(since the first official C++ standard in 1998) and compilers
are starting to ship without it. The replacement is <iostream>
(without the ending ".h"), but you'll almost certainly have to
add a, "using namespace std;" somewhere in source files that
include it, as code that's old enough to use <iostream.h> is
probably also old enough to predate the widespread use of C++
namespaces.

I'd try pulling the headers off of the command line first to
see if that helps.

If the tarball of the file in question is available somewhere,
I might be able to pull of a few cycles to see if I can get it
to compile.

Hmm. There really ought to be a Github organization or
something for ancient BBS software updated to compile on
modern systems.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

tenser wrote (2021-01-18):

The docs say there should be a suggested make shell script for gcc
with it, but this seems to be missing in the zip.

There is no `makefile` or `Makefile`? That's unfortunate.

makespoon which is just

g++ spoon.cpp config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp -o spoon

2) <iostream.h> has itself been deprecated for almost 25 years
(since the first official C++ standard in 1998) and compilers
are starting to ship without it. The replacement is <iostream>
(without the ending ".h"), but you'll almost certainly have to
add a, "using namespace std;" somewhere in source files that
include it, as code that's old enough to use <iostream.h> is
probably also old enough to predate the widespread use of C++
namespaces.

I think this is what I did some years ago. See https://gitlab.com/fidosoft/soupgate

I still get warnings like:

$ ./makespoon
spoon.cpp:60:7: warning: built-in function ‘logf’ declared as non-function [-Wbuiltin-declaration-mismatch]
FILE *logf = NULL;
^~~~
spoon.cpp: In function ‘void checkdir(char*)’:
spoon.cpp:143:69: warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings]
logprintf("FATAL ERROR: subdirectory %s does not exist\n", dir);

but it compiles and run.

Hmm. There really ought to be a Github organization or
something for ancient BBS software updated to compile on
modern systems.

That's a good idea

There is some software at https://github.com/ftnapps (which should compile)

some I collected at https://github.com/fidosoft and https://gitlab.com/fidosoft, but many don't compile.

It would be good to have people available who can help to get things build again. I think most of the time the errors are not very hard to fix, if you know C(++) and know what's changed over the years. For people without C experience (like me) it's sometimes hard to figure out how to fix it properly without introducing new bugs.

Are there any limitations for organizations on the free github plan or is it good enough for public repos? (I only remember that I could do more with gitlab some years ago, but this was before Microsoft took over and was maybe related to private repos).

---
* Origin: this message must NOT be gated to Telegram (21:3/102)

On 18 Jan 2021 at 10:43a, Oli pondered and said...

makespoon which is just

g++ spoon.cpp config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp
socket.cpp -o spoon

I see; they distributed it as a shell script. It seems like
that may be missing from Paul's copy of the archive, though.

I think this is what I did some years ago. See https://gitlab.com/fidosoft/soupgate

I still get warnings like:

$ ./makespoon
spoon.cpp:60:7: warning: built-in function ‘logf’ declared as non-function [-Wbuiltin-declaration-mismatch]
FILE *logf = NULL;
^~~~

Ah, this is due to an unfortunate name. `logf` is the
name of a function from the standard math library
(single-precision floating point natural logarithm),
and as per the standard, names of standard functions
are reserved. That's also untenable for names like
`log`, hence why this is a warning, not an error.
Simply renaming the variable would eliminate that,
though.

spoon.cpp: In function ‘void checkdir(char*)’:
spoon.cpp:143:69: warning: ISO C++ forbids converting a string constant
to ‘char*’ [-Wwrite-strings]
logprintf("FATAL ERROR: subdirectory %s does not exist\n", dir);

but it compiles and run.

This is a single type error. `logprintf` is probably
defined to take `char *` as its first argument; this
warning is saying that one is passing a string constant
as that argument, but that the function is defined so
that it could mutate the thing it points to. The fix
here is to add a `const` qualifier to the argument.

It would be good to have people available who can help to get things
build again. I think most of the time the errors are not very hard to
fix, if you know C(++) and know what's changed over the years. For
people without C experience (like me) it's sometimes hard to figure out how to fix it properly without introducing new bugs.

Yeah, most of this stuff is pretty simple. A lot of it
is just that the code predates the relevant standards, or
has obvious bugs.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

tenser wrote (2021-01-19):

I still get warnings like:

$ ./makespoon
spoon.cpp:60:7: warning: built-in function ‘logf’ declared as
non-function [-Wbuiltin-declaration-mismatch]
FILE *logf = NULL;
^~~~

Ah, this is due to an unfortunate name. `logf` is the
name of a function from the standard math library
(single-precision floating point natural logarithm),

that worked.

spoon.cpp: In function ‘void checkdir(char*)’:
spoon.cpp:143:69: warning: ISO C++ forbids converting a string
constant to ‘char*’ [-Wwrite-strings]
logprintf("FATAL ERROR: subdirectory %s does not exist\n",
dir);

but it compiles and run.

This is a single type error. `logprintf` is probably
defined to take `char *` as its first argument; this
warning is saying that one is passing a string constant
as that argument, but that the function is defined so
that it could mutate the thing it points to. The fix
here is to add a `const` qualifier to the argument.

void logprintf(const char *s, ...);

compiles, but so does

void logprintf(char const *s, ...);

is it the same?

---
* Origin: this message must NOT be gated to Telegram (21:3/102)

On 18 Jan 2021 at 04:49p, Oli pondered and said...

void logprintf(const char *s, ...);

compiles, but so does

void logprintf(char const *s, ...);

Yup. Those are the same.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 18 Jan 2021 at 02:14a, tenser pondered and said...

gcc config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp socket.hpp sock.h spoon.cpp spoon.h -o spoon.test

Generally speaking, you don't try to compile header files
(.hpp or .h) directly. Rather, those are usually meant to
be "included" in a source file that is then compiled.

Thanks, yep even to this untrained bloke it seemed like when I looked at the errors there were things being called that were missing. I had a feeling not all files needed to be included in the command line but also no idea of what
I should or should not use. :)

Running `gcc` by itself may well be just fine, as it's

user-level programs...I digress). But running g++ or
c++ is also fine and certainly won't hurt anything.

Yep I tried both but no joy.

There is no `makefile` or `Makefile`? That's unfortunate.

No the docs said there would be but no dice.

I'd try pulling the headers off of the command line first to
see if that helps.

If the tarball of the file in question is available somewhere,
I might be able to pull of a few cycles to see if I can get it
to compile.

I have not looked / played with it in a few days now but thanks for the offer.

I did try a version that Oli passed to me via a git repo but I have not yet
had success getting soupgate and soup to work totally correctly as yet for
any static or complied versions I have managed to build or obtain as yet.

http://software.tomsweb.net/soupgate.html has the links to the static files
and linux source.

So far under linux (and I'm a bit woolly on this now as it's been a few days)

I have managed to get (I think it was Oli's ) version of Soup to poll my NNTP server and pull down some packets, then I think it was the static soupgate
file that gated them to PKT, then (and lordy I have got myself a bit muddled now with all the versions I have been trying) it was Oli's soupgate that
would at least see the PKT to be exported in the FTN outbound, the static version didn't see them, and I (only once) got I think his version of soup to post to NNTP but then it borked after the NNTP bit as files on the HDD that should have been removed were not. Phew..

You'll understand why I have not been back at this in few days.

At the heart of all this is my need to find Linux alternatives to what I currently run on Win32 to parse FTN PKTs in/out of my NNTP server. For
windows I have been using vSoup but that does not exist for Linux as far as I can tell hence playing with the authors Soup software to see if that would work.

I may to start with end up having to fudge things between two systems linux
and windows in order to get gating working while starting to run Linux for
HUB packet tossing duties.

To start with I plan to cut over things that handle my Fido and othernets traffic that I carry but not fsxNet as yet... I really want to sort things
out using the othernets first before I look to move 21:1/100 over as well..

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Re: Re: Compiling from source
By: Avon to tenser on Thu Jan 21 2021 08:21 pm

Hey,

At the heart of all this is my need to find Linux alternatives to what I currently run on Win32 to parse FTN PKTs in/out of
my NNTP server. For

A thought - you should be able to get Syncronet being your NNTP to FTN gateway. It works well (I'm using it to get some newsgroups from eternal september).

I'm already feeding a couple of folks with it (can feed anybody else if there is interest).

...ëîåï

... A triangle which has an angle of 135 degrees is called an obscene triangle --- SBBSecho 3.11-Linux
* Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116)

On 21 Jan 2021 at 10:24p, deon pondered and said...

A thought - you should be able to get Syncronet being your NNTP to FTN gateway.

It works well (I'm using it to get some newsgroups from eternal september).

OK thanks, yeah I'll keep it up my sleeve as a plan B, C or D :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-22):

On 21 Jan 2021 at 10:24p, deon pondered and said...

A thought - you should be able to get Syncronet being your NNTP to
FTN gateway.

It works well (I'm using it to get some newsgroups from eternal
september).

OK thanks, yeah I'll keep it up my sleeve as a plan B, C or D :)

There is also ifgate, fidogate, watergate as plan E, F, ...

---
* Origin: this message must NOT be gated to Telegram (21:3/102)

On 22 Jan 2021 at 08:35a, Oli pondered and said...

There is also ifgate, fidogate, watergate as plan E, F, ...

It might yet come to that. I wish I weren't so darn tired tonight.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-22):

On 22 Jan 2021 at 08:35a, Oli pondered and said...

There is also ifgate, fidogate, watergate as plan E, F, ...

It might yet come to that.

Just in case, there is a fidogate fork with regular updates on github: https://github.com/ykaliuta/fidogate

I never tried it, but I guess if the simpler solutions don't work, it could be done with fidogate.

Watergate looked really nice, but the last release is pretty old. It would be a good candidate for porting to a recent FreePascal / Linux though ;)

I wish I weren't so darn tired tonight.

I wish I weren't so tired this morning :)

* Origin: this message must NOT be gated to Telegram (21:3/102)

On 22 Jan 2021 at 09:53a, Oli pondered and said...

Just in case, there is a fidogate fork with regular updates on github: https://github.com/ykaliuta/fidogate

I've spent a further 3 hours today trying to get various versions of soupgate for linux to work correctly and also soup.

What I found was

oli soup - will compile and seems to happily pass login/pass to NNTP and then pull down / create soup packets for newsgroups.

oli soupgate import failed with errors, saying it couldn't read the files it was trying to import

I then used the static compiled soupgate.static from the website to run
import and that worked to created FTN packets the HUB could import

Testing the otherway from HUB to gate to NNTP has been the real issue.

oli soupgate export worked but oli spoon failed to post to NNTP reporting issues in the files it was trying to post

* Posting article to nz.test
NNTP host returned error: 441 Can't parse Date: header
! NNTP server did not accept message
free(): double free detected in tcache 2
Aborted

I reset the export test and used soupgate.static export but it failed to see the FTN packet in the outbound directory despite it being present :(

On a hunch I changed the .clo file in the outbound to .CLO and then the soupgate.static found the packet and exported it.

Then I tried using oli soup to send to NNTP and while it posted it still also fully failed to run correctly borking with this error

* Posting article to nz.test
free(): double free detected in tcache 2
Aborted

The REPLIES file in my Soup folder was still left behind, I think related to this abort.

In sum,

3 hours I can't get back :)
Seems problematic to get two linux versions of files working.
No idea why the static soupgate seems so picky about exporting files and only happy to look for upper case BSO files.. sigh.

I may well soon turn attention to researching and trying to get another
option working for linux.

I want to get gateway stuff sorted before putting the Fido HUB with HPT
online.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 22 Jan 2021 at 09:53a, Oli pondered and said...

Watergate looked really nice, but the last release is pretty old. It
would be a

where do I find this one?

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-23):

I've spent a further 3 hours today trying to get various versions of soupgate for linux to work correctly and also soup.

[...]

In sum,

3 hours I can't get back :)
Seems problematic to get two linux versions of files working.

Could it be some kind of 64-bit problem? If I find the time I will try soupgate on 32bit and 64bit.

I may well soon turn attention to researching and trying to get another option working for linux.

It's not unlikely that fidogate will get you a running and stable system faster than soupgate, even it takes some time to learn and configure it. But I have never used it, just guessing.

I've installed Cyrus as an IMAP and NNTP server and I'm also interested to play around with FTN<->NNTP gating.

---
* Origin: . (21:3/102)

Avon wrote (2021-01-23):

On 22 Jan 2021 at 09:53a, Oli pondered and said...

Watergate looked really nice, but the last release is pretty old. It
would be a

where do I find this one?

https://sourceforge.net/projects/watergate

---
* Origin: . (21:3/102)

On 23 Jan 2021 at 10:02a, Oli pondered and said...

Could it be some kind of 64-bit problem? If I find the time I will try soupgate

on 32bit and 64bit.

I'm running Debian Buster X86_64 on this box.

If there's something I need to do to compile the source differently or ? I'd
be open to ideas. Thanks :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 23 Jan 2021 at 10:02a, Oli pondered and said...

It's not unlikely that fidogate will get you a running and stable system faster
than soupgate, even it takes some time to learn and configure it. But I have never used it, just guessing.

It seems like quite a beast in that it also offers not just gating but
tossing etc. as well. I'm well down the path of setting up HPT as a tosser so not really wanting to use that aspect of Fidogate if I don't need to. But who knows.

It's a bit disheartening trying to get what is a simple process involving two exe files in windows working on Linux.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-24):

On 23 Jan 2021 at 10:02a, Oli pondered and said...

It's not unlikely that fidogate will get you a running and stable
system faster
than soupgate, even it takes some time to learn and configure it.
But I have never used it, just guessing.

It seems like quite a beast in that it also offers not just gating but tossing etc. as well. I'm well down the path of setting up HPT as a
tosser so not really wanting to use that aspect of Fidogate if I don't
need to. But who knows.

:)

I think fidogate does not support message bases like Squish or JAM. Only pass-through tossing and gating to/from an NNTP server.

It's a bit disheartening trying to get what is a simple process involving two exe files in windows working on Linux.

Yeah ... FTN / BBS is really a niche and most software is unmaintained.

---
* Origin: . (21:3/102)

Avon wrote (2021-01-24):

On 23 Jan 2021 at 10:02a, Oli pondered and said...

Could it be some kind of 64-bit problem? If I find the time I will
try soupgate

on 32bit and 64bit.

I'm running Debian Buster X86_64 on this box.

If there's something I need to do to compile the source differently or ? I'd be open to ideas. Thanks :)

there are a bunch of variables defined as "long", which I think are 64-bit integers on 64-bit Linux and 32-bit on 32-bit Linux. I'm not sure that it makes a difference for soupgate/spoon, but you could try the -m32 parameter with gcc/g++

https://www.geeksforgeeks.org/compile-32-bit-program-64-bit-gcc-c-c/

---
* Origin: . (21:3/102)

Oli wrote (2021-01-24):

on 32bit and 64bit.

I'm running Debian Buster X86_64 on this box.

If there's something I need to do to compile the source differently
or ? I'd be open to ideas. Thanks :)

there are a bunch of variables defined as "long", which I think are
64-bit integers on 64-bit Linux and 32-bit on 32-bit Linux. I'm not sure that it makes a difference for soupgate/spoon, but you could try the -m32 parameter with gcc/g++

found the first problem in pktview. It doesn't show the correct header correct in the 64-bit version. We should assume the 64-bit build is broken for soupgate and spoon too until otherwise proven.

---
* Origin: . (21:3/102)

Oli wrote (2021-01-24):

found the first problem in pktview. It doesn't show the correct header correct in the 64-bit version. We should assume the 64-bit build is
broken for soupgate and spoon too until otherwise proven.

32-bit (Raspi) - correct
--- PACKED MESSAGE #1 ---
Version : 2
Originating net : 2
Originating node : 116
Destination net : 3
Destination node : 102
Cost : 0
Attributes : Pvt
Date and time : 24 Jan 21 21:07:42

64-bit (amd64) - all wrong
--- PACKED MESSAGE #1 ---
Version : 102
Originating net : 1
Originating node : 2
Destination net : 0
Destination node : 3
Cost : 18976
Attributes : Cra F/A I/T (?) Rrq Cpt
Date and time : an 21 21:07:42

---
* Origin: . (21:3/102)

Avon wrote (2021-01-24):

On 23 Jan 2021 at 10:02a, Oli pondered and said...

Could it be some kind of 64-bit problem? If I find the time I will
try soupgate

on 32bit and 64bit.

I'm running Debian Buster X86_64 on this box.

If there's something I need to do to compile the source differently or ? I'd be open to ideas. Thanks :)

could you test, if a 32-bit compile works? pktview does work form me with "gcc -m32" even on 64-bit Debian:

apt install gcc-multilib g++-multilib

g++ -m32 spoon.cpp config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp -o spoon
gcc -m32 soupgate.c config.c lists.c pkt.c import.c export.c -o soupgate
gcc -m32 pktview.c -o pktview

If it does work, maybe we could port soupgate and tools to 64-bit.

---
* Origin: . (21:3/102)

On 24 Jan 2021 at 04:14p, Oli pondered and said...

could you test, if a 32-bit compile works? pktview does work form me
with "gcc -m32" even on 64-bit Debian:

Just so I'm clear, I need to be running a 32bit version of Linux to do the compile and testing?

Or are you asking me to just run the compile commands on Debian 64bit and
then see if it works?

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-25):

On 24 Jan 2021 at 04:14p, Oli pondered and said...

could you test, if a 32-bit compile works? pktview does work form me
with "gcc -m32" even on 64-bit Debian:

Just so I'm clear, I need to be running a 32bit version of Linux to do the compile and testing?

no, you don't.

Or are you asking me to just run the compile commands on Debian 64bit and then see if it works?

yes

apt install gcc-multilib g++-multilib

installs the necessary 32-bit libraries (on a 64-bit system)

"gcc -m32" compiles a 32-bit binary even if the OS is 64-bit.

I can also send you the built binaries. They have been compiled on Debian Testing (Bullseye), but I think they should work.

---
* Origin: . (21:3/102)

On 25 Jan 2021 at 07:43a, Oli pondered and said...

Just so I'm clear, I need to be running a 32bit version of Linux to do compile and testing?

no, you don't.

10-4

Or are you asking me to just run the compile commands on Debian 64bit then see if it works?

yes

Gotcha.

Is this using the main source files from the website or using your git repo?

I can also send you the built binaries. They have been compiled on Debian Testing (Bullseye), but I think they should work.

Yep feel free to drop them in a filebox to 21:1/100 if able.

I'll try and do those tests over the coming day.

Just doing admin stuff at present. There's always admin to do. You'll love it they said, it'll be fun they said.. :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-25):

10-4

I had to look that up :)

Is this using the main source files from the website or using your git repo?

what is the URL of the website anyway? I don't remember where I downloaded soupgate.

you need the fixes in the git repo.

I can also send you the built binaries. They have been compiled on
Debian Testing (Bullseye), but I think they should work.

Yep feel free to drop them in a filebox to 21:1/100 if able.

ok. I'll send a zip.

---
* Origin: . (21:3/102)

On 25 Jan 2021 at 09:05a, Oli pondered and said...

what is the URL of the website anyway? I don't remember where I
downloaded soupgate.

http://software.tomsweb.net/soupgate.html

you need the fixes in the git repo.

OK, I guess I should clone it again to be sure I get all the good stuff :)

ok. I'll send a zip.

Thanks

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-25):

what is the URL of the website anyway? I don't remember where I
downloaded soupgate.

http://software.tomsweb.net/soupgate.html

Good. The sgsources.zip is the same as the one in the gitlab repo. I wasn't sure that I had used the latest release.

ok. I'll send a zip.

done.

---
* Origin: . (21:3/102)

On 25 Jan 2021 at 11:18a, Oli pondered and said...

ok. I'll send a zip.

done.

It's been a bit of mare of an evening. The box I am using to setup this HUB would not post. So I had to phaff with the graphics card for a while.

I then copied the zip contents over to it and none of them seemed to want to run.

I'm just pulling down the repo again and will try compiling them on the box

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 24 Jan 2021 at 04:14p, Oli pondered and said...

apt install gcc-multilib g++-multilib

g++ -m32 spoon.cpp config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp
-o spoon
gcc -m32 soupgate.c config.c lists.c pkt.c import.c export.c -o soupgate gcc -m32 pktview.c -o pktview

all three compiled.

spoon appeared to work puling down messages from NNTP.

--- Spoon-Linux v1.06 Tue 26-Jan-2021 20:59:34
Fetching news from xxx.xxx.xxx.xxx
* Fetching alt.bbs.mystic
There are 1 new messages (5896-5896) in this newsgroup.
* Fetching comp.sys.cbm
There are 3 new messages (3098-3100) in this newsgroup.
* Fetching comp.sys.raspberry-pi
There are 26 new messages (16674-16699) in this newsgroup.
Average CPS fetching news: 143637
* Fetching nz.test
There are 5 new messages (200-204) in this newsgroup.
Average CPS fetching news: 21168

soupgate import failed and reported this error

--- SoupGate-Linux v1.06 Tue 26-Jan-2021 21:01:02
Importing from Soup to PKT
Importing alt.bbs.mystic
Error reading /hub/gateway/soup/00000000.msg
Importing comp.sys.cbm
Error reading /hub/gateway/soup/00000001.msg
Importing comp.sys.raspberry-pi
Error reading /hub/gateway/soup/00000002.msg
Importing nz.test
Error reading /hub/gateway/soup/00000003.msg
No inbound messages found. Nothing to do.

at the end of running import it appears to delete the AREAS file that s

On 26 Jan 2021 at 09:11p, Avon pondered and said...

I have not yet tested export for soupgate nor posting the exported files (assuming they work)

I tested an export of a packet from HPT and this seemed to work

Exporting from ?UT/?LO to Soup
Exporting Type 2+ packet from 3:770/1 to 3:770/3

1 messages exported

mail: 0 news: 1

This was an old packet I had posted to the NNTP server so it was interesting
to test this also

Spoon did try to post the packet, NNTP (rightfully) said No.

Posting news to xxx.xxx.xxx.xxx
* Posting article to nz.test
NNTP host returned error: 441 435 Duplicate
! NNTP server did not accept message
free(): double free detected in tcache 2
Aborted


Despite the failed post Spoon is not exiting correctly. This has been the
same error I have seen in all my test compiles.

free(): double free detected in tcache 2
Aborted

Not sure what that is or how to fix, but I can report when Soupgate creates
an exported message it generates a number of .MSG files holding the gated
posts to be sent to the NNTP and a REPLIES file that acts like an index of
all the .MSG files to be sent.

On a successful post session using Spoon the REPLIES should be deleted. But much like the error with Soupgate importing messages the REPLIES file
remained undeleted.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 26 Jan 2021 at 09:21p, Avon pondered and said...

I tested an export of a packet from HPT and this seemed to work

Exporting from ?UT/?LO to Soup
Exporting Type 2+ packet from 3:770/1 to 3:770/3

1 messages exported

mail: 0 news: 1

I re-ran this test again with a fresh packet to export, all seemed fine

I then ran soup to post it and it did post, but again this error and the REPLIES file is left behind.


Posting news to xxx.xxx.xxx.xxx
* Posting article to nz.test
free(): double free detected in tcache 2
Aborted


So it seems like there an issue with

Soup - posting messages
Soupgate - importing messages

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-26):

spoon appeared to work puling down messages from NNTP.

--- Spoon-Linux v1.06 Tue 26-Jan-2021 20:59:34
Fetching news from xxx.xxx.xxx.xxx
* Fetching alt.bbs.mystic
There are 1 new messages (5896-5896) in this newsgroup.

[...]

soupgate import failed and reported this error

--- SoupGate-Linux v1.06 Tue 26-Jan-2021 21:01:02
Importing from Soup to PKT
Importing alt.bbs.mystic
Error reading /hub/gateway/soup/00000000.msg

[...]

No inbound messages found. Nothing to do.

Could you send me (or put on hold) a zip with some Soup messages so I can try to import them and see if I find the error?

at the end of running import it appears to delete the AREAS file that
spoon created..

It looks soupgate doesn't handle errors not always gracefully.

I can also see three 0kb files created when I ran the import called

00000000.TMP
00000001.TMP
00000002.TMP

I wonder why it cant read the files it's trying to import.

me too :)

I have not yet tested export for soupgate nor posting the exported files (assuming they work)

I successfully exported a PKT file to Soup, at least the Soup files looked fine (as far as I can tell). But my soupgate doesn't find anything, if I use the /pkt=pktdir parameter.

---
* Origin: . (21:3/102)

On 26 Jan 2021 at 09:43a, Oli pondered and said...

Importing from Soup to PKT
Importing alt.bbs.mystic
Error reading /hub/gateway/soup/00000000.msg

[...]

No inbound messages found. Nothing to do.

Could you send me (or put on hold) a zip with some Soup messages so I
can try to import them and see if I find the error?

Will do. This will be tomorrow night (my time) as I'm just heading to bed now but sure thing :) I'll send you the .msg files and the other index file soup creates.

at the end of running import it appears to delete the AREAS file that spoon created..

It looks soupgate doesn't handle errors not always gracefully.

Yep. I think it's correct to remove AREAS but ideally after it's used it to figure out which .msg files import to which area

I can also see three 0kb files created when I ran the import called

00000000.TMP
00000001.TMP
00000002.TMP

I wonder why it cant read the files it's trying to import.

me too :)

These look to be working files created in the same directory I ran soupgate import from. There is a variable in the config for a temp directory

TempDir

But I suspect it was ignoring that at the time of import?

I successfully exported a PKT file to Soup, at least the Soup files
looked fine (as far as I can tell). But my soupgate doesn't find
anything, if I use the /pkt=pktdir parameter.

Make sure you export something from a tosser with a .bso file like a .clo
that accompanies the PKT then running 'soupgate export' should work. Don't
use .hlo files

I too have had no joy with linux /pkt= switch, but it works fine for the windows version from the website. perhaps it's a pathing thing in the code or
a case thing?

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-26):

Posting news to xxx.xxx.xxx.xxx
* Posting article to nz.test
NNTP host returned error: 441 435 Duplicate
! NNTP server did not accept message
free(): double free detected in tcache 2

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Aborted

Despite the failed post Spoon is not exiting correctly. This has been the same error I have seen in all my test compiles.

I think we need help from a Professional

---
* Origin: . (21:3/102)

On 26 Jan 2021 at 09:21p, Avon pondered and said...

[snip]

Spoon did try to post the packet, NNTP (rightfully) said No.

Posting news to xxx.xxx.xxx.xxx
* Posting article to nz.test
NNTP host returned error: 441 435 Duplicate
! NNTP server did not accept message
free(): double free detected in tcache 2
Aborted

Sorry, I haven't been keeping up here, but this is a bug
in that software: the code is simply mismanaging memory.
The solution here is to find the bug and fix it; sadly,
much of this stuff is an unmaintained mess.... But trying
a 32-bit build won't fix it, as was suggested earlier.

Despite the failed post Spoon is not exiting correctly. This has been the same error I have seen in all my test compiles.

free(): double free detected in tcache 2
Aborted

Not sure what that is or how to fix, but I can report when Soupgate creates an exported message it generates a number of .MSG files holding the gated posts to be sent to the NNTP and a REPLIES file that acts like an index of all the .MSG files to be sent.

On a successful post session using Spoon the REPLIES should be deleted. But much like the error with Soupgate importing messages the REPLIES file remained undeleted.

Well, a thing to try is to build it with one of the memory
sanitizers turned on and see if you can find the location
of the first free and the second; simply nil'ing out the pointer
will likely move past _that_ problem, as free(NULL) is a no-op.
But a more general problem is that none of this stuff was written
to be memory safe; short of a substantial refactoring of the
code, I suspect things like this will keep cropping up.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

tenser wrote (2021-01-27):

On 26 Jan 2021 at 09:21p, Avon pondered and said...

free(): double free detected in tcache 2
Aborted

Sorry, I haven't been keeping up here, but this is a bug
in that software: the code is simply mismanaging memory.
The solution here is to find the bug and fix it; sadly,
much of this stuff is an unmaintained mess.... But trying
a 32-bit build won't fix it, as was suggested earlier.

yes, these are separate issues, but building as 32-bit worked around some other issues.

Well, a thing to try is to build it with one of the memory
sanitizers turned on and see if you can find the location
of the first free and the second; simply nil'ing out the pointer
will likely move past _that_ problem, as free(NULL) is a no-op.

I couldn't see anything obvious in the spoon code. I found some examples for building with the sanitizer turned on. Would something like this work:

g++ spoon.cpp ... -o spoon -ggdb -fsanitize=address -fno-omit-frame-pointer

or a longer version (not sure if the libs are needed):

g++ spoon.cpp ... -o spoon -ggdb -fsanitize=address -fno-omit-frame-pointer -static-libstdc++ -static-libasan -lrt

But a more general problem is that none of this stuff was written
to be memory safe; short of a substantial refactoring of the
code, I suspect things like this will keep cropping up.

Rewrite everything in Go?

---
* Origin: . (21:3/102)

On 26 Jan 2021 at 09:43a, Oli pondered and said...

Could you send me (or put on hold) a zip with some Soup messages so I
can try to import them and see if I find the error?

* Fetching alt.bbs.mystic
There are 100 new messages (5797-5896) in this newsgroup.
Average CPS fetching news: 49428
* Fetching comp.sys.cbm
There are 101 new messages (3001-3101) in this newsgroup.
Average CPS fetching news: 162655
* Fetching comp.sys.raspberry-pi
There are 103 new messages (16600-16702) in this newsgroup.
Average CPS fetching news: 249909

I have created a file called oli-soup-imported-news.tar.gz

Now how best to get it to you?

* Origin: . (21:3/102)

I'll netmail you with an idea.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 26 Jan 2021 at 10:32a, Oli pondered and said...

! NNTP server did not accept message
free(): double free detected in tcache 2

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Aborted

Despite the failed post Spoon is not exiting correctly. This has been same error I have seen in all my test compiles.

I think we need help from a Professional

A Google search revealed this info.

https://bit.ly/2YjfS5m

trying to assign l from an uninitialised array str

..and this link seemed helpful.

https://bit.ly/3cc3Ene

[snip]

Double free errors occur when free() is called more than once with the same memory address as an argument.

Calling free() twice on the same value can lead to memory leak. When a
program calls free() twice with the same argument, the program's memory management data structures become corrupted and could allow a malicious user
to write values in arbitrary memory spaces. This corruption can cause the program to crash or, in some circumstances, alter the execution flow. By overwriting particular registers or memory spaces, an attacker can trick the program into executing code of their own choosing, often resulting in an interactive shell with elevated permissions.

When a buffer is free()'d, a linked list of free buffers is read to rearrange and combine the chunks of free memory (to be able to allocate larger buffers
in the future). These chunks are laid out in a double linked list which
points to previous and next chunks. Unlinking an unused buffer (which is what happens when free() is called) could allow an attacker to write arbitrary values in memory; essentially overwriting valuable registers, calling
shellcode from its own buffer.

[snip]

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 27 Jan 2021 at 02:59a, tenser pondered and said...

Sorry, I haven't been keeping up here, but this is a bug
in that software: the code is simply mismanaging memory.
The solution here is to find the bug and fix it; sadly,
much of this stuff is an unmaintained mess.... But trying
a 32-bit build won't fix it, as was suggested earlier.

I confess I'm mostly stumbling around in the dark here hoping something will work. My preference is to try to get the Linux code working if doable so as
to be able to keep using what (until now) has been a relatively straight forward bit of software (for windows) and not have to figure out another more bulky option I have yet to find.

I think the 32bit thing may have solved some of the issues... but problems remain with both soupgate import and soup export/posting to NNTP. :(

On a successful post session using Spoon the REPLIES should be delete But much like the error with Soupgate importing messages the REPLIES remained undeleted.

Well, a thing to try is to build it with one of the memory
sanitizers turned on and see if you can find the location
of the first free and the second; simply nil'ing out the pointer
will likely move past _that_ problem, as free(NULL) is a no-op.
But a more general problem is that none of this stuff was written
to be memory safe; short of a substantial refactoring of the
code, I suspect things like this will keep cropping up.

Thank you for the guidance. I think Oli is perusing this and I'm trying to
get him some news / soup files to play with.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-27):

On 26 Jan 2021 at 09:43a, Oli pondered and said...

Could you send me (or put on hold) a zip with some Soup messages so
I can try to import them and see if I find the error?

* Fetching alt.bbs.mystic
There are 100 new messages (5797-5896) in this newsgroup.
Average CPS fetching news: 49428
* Fetching comp.sys.cbm
There are 101 new messages (3001-3101) in this newsgroup.
Average CPS fetching news: 162655
* Fetching comp.sys.raspberry-pi
There are 103 new messages (16600-16702) in this newsgroup.
Average CPS fetching news: 249909

I have created a file called oli-soup-imported-news.tar.gz

SUCCESS!!!

../soupgate import
SoupGate-Linux v1.06 - Fidonet/Internet gateway software
Copyright (c) 1997-2000 by Tom Torfs, all rights reserved

Beta version; compiled Jan 27 2021 10:21:11

This is FREEWARE. Read soupgate.doc for more information.

Importing from Soup to PKT

Importing alt.bbs.mystic

100 message(s) imported

mail: 0 news: 100 lists: 0 junk: 0
personal: 0 own org: 0 files: 0 ign/txt: 0
Importing comp.sys.cbm

101 message(s) imported

mail: 0 news: 101 lists: 0 junk: 0
personal: 0 own org: 0 files: 0 ign/txt: 0
Importing comp.sys.raspberry-pi

103 message(s) imported

mail: 0 news: 103 lists: 0 junk: 0
personal: 0 own org: 0 files: 0 ign/txt: 0

---
* Origin: . (21:3/102)

On 27 Jan 2021 at 10:30a, Oli pondered and said...

SUCCESS!!!

103 message(s) imported

mail: 0 news: 103 lists: 0 junk: 0

That's good stuff, if I pull down the latest version of the repo and compile it, do you think it would work for me too?

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-28):

On 27 Jan 2021 at 10:30a, Oli pondered and said...

SUCCESS!!!

103 message(s) imported

mail: 0 news: 103 lists: 0 junk: 0

That's good stuff, if I pull down the latest version of the repo and compile it, do you think it would work for me too?

Yes, I forgot to mention that.

---
* Origin: . (21:3/102)

On 28 Jan 2021 at 08:50a, Oli pondered and said...

That's good stuff, if I pull down the latest version of the repo and compile it, do you think it would work for me too?

Yes, I forgot to mention that.

Coolio, thanks Oli. I'll give it a try.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-28):

On 28 Jan 2021 at 08:50a, Oli pondered and said...

That's good stuff, if I pull down the latest version of the
repo and compile it, do you think it would work for me too?

Yes, I forgot to mention that.

Coolio, thanks Oli. I'll give it a try.

I just committed and pushed some 64-bit fixes. It might now work without the -m32 flag too.

---
* Origin: . (21:3/102)

On 27 Jan 2021 at 02:59a, tenser pondered and said...

free(): double free detected in tcache 2
Aborted

Well, a thing to try is to build it with one of the memory
sanitizers turned on and see if you can find the location
of the first free and the second; simply nil'ing out the pointer
will likely move past _that_ problem, as free(NULL) is a no-op.
But a more general problem is that none of this stuff was written

Could you guide me as to how to do this?

I'm looking at the man for g++ but so far unsure how to enable a sanitizer mode.

I'll keep poking and prodding :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 28 Jan 2021 at 10:39a, Oli pondered and said...

I just committed and pushed some 64-bit fixes. It might now work without the -m32 flag too.

compiled the latest versions of soup and soupgate tonight

I could pull down news and import it to PKT and could also export PKT back to news... only Soup News when posting is borking with the double free memory error after it successfully posts the gated news to the NNTP server.

I spent some hours trying to learn what I might be able to do to debug

Came across a tool called Clang that has a -fsanitize=address option

I'm not really sure what I'm doing with this one :)


clang -fsanitize=address -O1 -fno-omit-frame-pointer -g spoon.cpp config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp
pop.cpp:294:71: warning: format specifies type 'int' but the argument has
type 'long' [-Wformat]
printf("Examining headers of message %d of %d...\n", i+1, nummessages);
~~ ^~~~~~~~~~~
%ld
pop.cpp:382:22: warning: format specifies type 'int' but the argument has
type 'long' [-Wformat]
i+1, nummessages, msglist[i].msgsize);
^~~~~~~~~~~
2 warnings generated.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-29):

Came across a tool called Clang that has a -fsanitize=address option

That's a C compiler.

I'm not really sure what I'm doing with this one :)

clang -fsanitize=address -O1 -fno-omit-frame-pointer -g spoon.cpp config.cpp keyboard.cpp nntp.cpp pop.cpp smtp.cpp socket.cpp pop.cpp:294:71: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
printf("Examining headers of message %d of %d...\n", i+1, nummessages);
~~
^~~~~~~~~~~
%ld
pop.cpp:382:22: warning: format specifies type 'int' but the argument has type 'long' [-Wformat]
i+1, nummessages, msglist[i].msgsize);
^~~~~~~~~~~
2 warnings generated.

Compile time warnings. Easy to fix, but will not help with the NNTP bug. pop.cpp is for POP3 mail fetch.

---
* Origin: . (21:3/102)

On 29 Jan 2021 at 08:43p, Oli pondered and said...

Came across a tool called Clang that has a -fsanitize=address option

That's a C compiler.

Oh OK, yeah I was trying to find a tool to follow up on Tensers suggestion of looking for the double free memory issue.

I can understand how it can be a pain to try and track down as (if I
understand this right) we're essentially looking for locations across the
code where a variable is being called again but has already been
closed/nulled? It seems like there are tools about that can check the code on compile and alert to hints as to where/what may be going on?

2 warnings generated.

Compile time warnings. Easy to fix, but will not help with the NNTP bug. pop.cpp is for POP3 mail fetch.

Yep that's how I interpreted it also... interesting to see the issues in the pop code, not that I use it... I've been trying to understand the nntp code
and the area around when a post is made and then the error occurs.

I'm a 2 year old trying to read a scientific novel :)

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

Avon wrote (2021-01-30):

On 29 Jan 2021 at 08:43p, Oli pondered and said...

Came across a tool called Clang that has a -fsanitize=address
option

That's a C compiler.

Oh OK, yeah I was trying to find a tool to follow up on Tensers
suggestion of looking for the double free memory issue.

(of course Clang is also a C++ and Objective C compiler)

I can understand how it can be a pain to try and track down as (if I understand this right) we're essentially looking for locations across the code where a variable is being called again but has already been closed/nulled? It seems like there are tools about that can check the
code on compile and alert to hints as to where/what may be going on?

The compiler is the right tool for this. If I understand it correctly you would get the warnings / hints at runtime. Did you try to run the clang compiled spoon

Compile time warnings. Easy to fix, but will not help with the NNTP
bug. pop.cpp is for POP3 mail fetch.

Yep that's how I interpreted it also... interesting to see the issues in the pop code, not that I use it... I've been trying to understand the
nntp code and the area around when a post is made and then the error occurs.

I think it is in the socket.cpp code. There are a couple of deletes.

---
* Origin: . (21:3/102)

Avon wrote (2021-01-29):

2 warnings generated.

fixed. see the changes with

git diff a3eac^ a3eac

---
* Origin: . (21:3/102)

BY: Oli(21:3/102)


|11O|09> |10Avon wrote (2021-01-29):|07
|11O|09> |07
|11O|09> |10A> 2 warnings generated.|07
|11O|09> |07
|11O|09> |10fixed. see the changes with|07
|11O|09> |07
|11O|09> |10git diff a3eac^ a3eac|07

What's the git repo for these changes (I tried searching through the messages but didn't find it).

Also which pktview are you using? I'll try to fix that for 64-bit. |11r|09ushfan|07


--- WWIV 5.6.0.3400
* Origin: Mystic Rhythms BBS (21:2/115)

Rushfan wrote (2021-01-30):

BY: Oli(21:3/102)

|11O|09>> |10Avon wrote (2021-01-29):|07
|11O|09>> |07
|10A>>> 2 warnings generated.|07
|11O|09>> |07
|11O|09>> |10fixed. see the changes with|07
|11O|09>> |07
|11O|09>> |10git diff a3eac^ a3eac|07

interesting quoting signs

What's the git repo for these changes (I tried searching through the messages but didn't find it).

https://gitlab.com/fidosoft/soupgate

Also which pktview are you using? I'll try to fix that for 64-bit.

Should be all fixed for 64-bit, same bug in soupgate and pktview. It was just one long in the packet header structure that must be 32-bit, but is 64-bit long on Linux.

---
* Origin: . (21:3/102)

On 30 Jan 2021 at 09:43p, Rushfan pondered and said...

What's the git repo for these changes (I tried searching through the messages but didn't find it).

If you are able to look at Soup and deduce the double free issue when
posting to NNTP that would be appreciated.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 29 Jan 2021 at 09:21p, Avon pondered and said...

On 27 Jan 2021 at 02:59a, tenser pondered and said...

free(): double free detected in tcache 2
Aborted

Well, a thing to try is to build it with one of the memory
sanitizers turned on and see if you can find the location
of the first free and the second; simply nil'ing out the pointer
will likely move past _that_ problem, as free(NULL) is a no-op.
But a more general problem is that none of this stuff was written

Could you guide me as to how to do this?

I'm looking at the man for g++ but so far unsure how to enable a
sanitizer mode.

I'll keep poking and prodding :)

Sure. First, perhaps it makes some sense to briefly describe
how we go from source code to an executable image in a Unix-style
environment, specifically using C/C++ compilers.

For most medium- to large-size programs, the basic gist of it is
that there are two logically separate steps:

1. Compilation. This is where we invoke the compiler on a
source file (e.g., .c/.cc/.cpp/whatever). The compiler
will read in the source file and, if it's a valid
compilable file, will emit an "object file": this is the
".o" file we see after running the compiler. Usually
the object file is in the "ELF" format.
2. Linking. This is where we invoke the linker on a set of
object files and library archives to produce an executable
image. The linker is responsible for combining object
files and resolving references between then and libraries.

Consider a simple program such as "Hello World": in C++, we
would write:

#include <iostream>

int
main()
{
std::cout << "Hello, World!" << std::endl;
return 0;
}

Suppose we put that text into "hello.cc". Then we can compile
it by running two commands:

% c++ -c hello.cc
% c++ -o hello hello.o

The first creates an object file called "hello.o". The second
invokes the linker to create an executable file called "hello"
that's created from whatever libraries the "c++" _driver command_
claims are necessary for C++ and from the "hello.o" object file.
For simple programs like this, we can invoke the compiler driver
program in such a way that the two steps are combined:

% c++ -o hello hello.cc

But note that these two independent steps still happen; I've used
the term "driver command" for the `c++` program, because that's
what it is: it's a program that actually invokes the real programs
that make up the compiler and linker.

Okay, why is any of this relevant? Because the sanitizers need
support from both the compiler _and_ the linker. You tell both
to enable sanitizer support by specifying command line flags to
the driver program: to enable address sanitizer (which will help
you find use-after-free and double-free bugs) you want to give
the `-fsanitize=address` argument. So I could try to compile
e.g. 'soup' as:

% c++ -g -fsanitize=address -o soup foo.cpp bar.cpp etc.cpp

In other words, edit the `makesoup` shell script and add
`-g -fsanitize=address` to the `g++` or `c++` line.

Then, when you run the resulting binary, it should tell you
where the double-free occurs.

This works since the `makesoup` shell script invokes the driver
program so that it compiles _and_ links the `soup` source files
in a single invocation.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)

On 01 Feb 2021 at 06:44a, tenser pondered and said...

In other words, edit the `makesoup` shell script and add
`-g -fsanitize=address` to the `g++` or `c++` line.

Then, when you run the resulting binary, it should tell you
where the double-free occurs.

Thanks :) I'll try to do this over the coming days. A bit time poor at
present and need to also add some new nodes to the network etc.

--- Mystic BBS v1.12 A46 2020/08/26 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)