• BBSINFO Data Requirements

    From stizzed@21:4/156 to All on Thu Jun 4 23:38:21 2020
    165
    All,

    Here is the latest compilation of information we have discussed including in the new BBSINFO DataFile (whatever form it takes). I am posting this to further the conversation about the standard:

    BBSINFO:

    Zone // BBS ZONE NUMBER (224)
    Net // BBS NET NUMBER (10)
    Node // BBS NODE NUMBER (0)
    BbsName // BBS NAME (My New BBS)
    City // BBS Location (City)
    State // BBS Location (State)
    Country // BBS Location (Country)
    Language // BBS Language (ENG, SPN, FRC, GMN, etc)
    SysopName // SYSOP NAME (Johnny SysOp)
    Phone // BBS PHONE NUMBER (888-555-1212)
    BBSURI1 // FULL BBS URL W/PORT (telnet://bbs.domain.tld:23)
    BBSURI2 // FULL BBS URL W/PORT (rlogin://bbs.domain.tld:513)
    BBSURI3 // FULL BBS URL W/PORT (www://www.bbs.domain.tld:80)
    BBSURI4 // FULL BBS URL W/PORT (ssh://bbs.domain.tld:22)
    Nodes // TOTAL # OF NODES SUPPORTED (DIALUP & TELNET)
    Callers // AVERAGE DAILY CALLERS TO THIS BBS (25)
    Online // BBS Operating Hours-GMT (0000-0000 or 0700-1800)
    Software // BBS SOFTWARE (Mystic v1.12 a45)
    Delete // FLAG FOR DELETION BY DOWNSTREAM SYSTEMS
    Verified // DATE INFO WAS VERIFIED (20200605)
    LognType // LOGIN TYPE (1=realname/2=handle/3=choic )
    cType // BBS TYPE (1=dialup/2=telnet/3=both

    Please bark if I forgot anything we have discussed and help us continue the conversation. Note that I have removed any reference to field lengths.
    Field lengths has been tabled until we determine what we want in the file.
    We will also settle on a format at that time.

    .\\ichael Batts
    a.k.a. stizzed (because, why not?)
    SysOp, The ROCK BBS III

    --- Mystic BBS v1.12 A45 2020/02/18 (Windows/32)
    * Origin: The ROCK III - therockbbs.net - TELNET:10023 (21:4/156)
  • From Ogg@21:4/106.21 to stizzed on Fri Jun 5 00:25:00 2020
    165
    Hello stizzed!

    ** On Thursday 04.06.20 - 23:38, stizzed wrote to All:

    Here is the latest compilation of information we have discussed including in the new BBSINFO DataFile (whatever form it takes)..

    How 'bout something to identify the different othernets the bbs would
    carry?


    --- OpenXP 5.0.44
    * Origin: Ogg's WestCoast Point (21:4/106.21)
  • From apam@21:1/126 to stizzed on Fri Jun 5 15:36:17 2020
    BBSURI1 // FULL BBS URL W/PORT (telnet://bbs.domain.tld:23)
    BBSURI2 // FULL BBS URL W/PORT (rlogin://bbs.domain.tld:513)
    BBSURI3 // FULL BBS URL W/PORT (www://www.bbs.domain.tld:80)
    BBSURI4 // FULL BBS URL W/PORT (ssh://bbs.domain.tld:22)

    I don't think this should be limited either, using any non ancient
    format could simply list an array of urls, and a bbs could have 1 - how
    ever many.

    Andrew


    --- MagickaBBS v0.15alpha (Linux/x86_64)
    * Origin: HappyLand - telnet://magickabbs.com:2023/ (21:1/126)
  • From stizzed@21:4/156 to Ogg on Fri Jun 5 07:57:41 2020
    165
    On |1505 Jun 2020|08, |15Ogg |08said the following...

    How 'bout something to identify the different othernets the bbs would carry?

    Yes, sorry! Avon suggested that and it was in my notes. Just got skipped over. Im posting an updated list which includes them as well as some
    questions about them.

    Thanks Ogg!

    .\\ichael Batts
    a.k.a. stizzed (because, why not?)
    SysOp, The ROCK BBS III

    --- Mystic BBS v1.12 A45 2020/02/18 (Windows/32)
    * Origin: The ROCK III - therockbbs.net - TELNET:10023 (21:4/156)
  • From stizzed@21:4/156 to apam on Fri Jun 5 07:59:23 2020
    165
    On |1505 Jun 2020|08, |15apam |08said the following...

    I don't think this should be limited either, using any non ancient
    format could simply list an array of urls, and a bbs could have 1 - how ever many.

    Andrew

    I (and many others) concur... In the latest list the suggestion of limits has been removed. Please see the latest list, to follow.

    Thanks Andrew!!!

    .\\ichael Batts
    a.k.a. stizzed (because, why not?)
    SysOp, The ROCK BBS III

    --- Mystic BBS v1.12 A45 2020/02/18 (Windows/32)
    * Origin: The ROCK III - therockbbs.net - TELNET:10023 (21:4/156)
  • From Warpslide@21:3/110 to apam on Fri Jun 5 09:13:00 2020
    On 05 Jun 2020, apam said the following...

    I don't think this should be limited either, using any non ancient
    format could simply list an array of urls, and a bbs could have 1 - how ever many.

    So instead of having BBSURI1-BBSURI4 we could just have one BBSURI and
    perhaps format it like:

    telnet://1.2.3.4:23;ssh://1.2.3.4:22;rlogin://1.2.3.4:513 - ?


    Jay

    --- Mystic BBS v1.12 A45 2020/02/18 (Windows/32)
    * Origin: Northern Realms BBS | bbs.nrbbs.net | Binbrook, ON (21:3/110)
  • From stizzed@21:4/156 to Warpslide on Fri Jun 5 09:41:29 2020
    165
    On |1505 Jun 2020|08, |15Warpslide |08said the following...

    On 05 Jun 2020, apam said the following...

    I don't think this should be limited either, using any non ancient format could simply list an array of urls, and a bbs could have 1 - h ever many.

    So instead of having BBSURI1-BBSURI4 we could just have one BBSURI and perhaps format it like:

    telnet://1.2.3.4:23;ssh://1.2.3.4:22;rlogin://1.2.3.4:513 - ?

    Yessir, we can certainly have that discussion. However, I think it prudent
    to wait until we determine the data we want to include before we decide on
    the best format (got my ass handed to me when I started the discussion with that) ;)

    .\\ichael Batts
    a.k.a. stizzed (because, why not?)
    SysOp, The ROCK BBS III

    --- Mystic BBS v1.12 A45 2020/02/18 (Windows/32)
    * Origin: The ROCK III - therockbbs.net - TELNET:10023 (21:4/156)
  • From NuSkooler@21:1/121 to stizzed on Fri Jun 5 18:26:42 2020
    Zone // BBS ZONE NUMBER (224)
    Net // BBS NET NUMBER (10)
    Node // BBS NODE NUMBER (0)
    BbsName // BBS NAME (My New BBS)
    City // BBS Location (City)
    State // BBS Location (State)
    Country // BBS Location (Country)
    Language // BBS Language (ENG, SPN, FRC, GMN, etc) SysopName // SYSOP BBSURI1 // FULL BBS URL W/PORT (telnet://bbs.domain.tld:23)
    BBSURI2... so on...

    Consider the following JSON (could be: XML (ew!!!), TOML, whatever):
    {
    "general" : {
    "boardName" : "Some BBS",
    // perhaps +op info, etc. fields here
    },
    "fidonet" : {
    "address" : "21:1/100" // FTN addrs are already parsable; could be up to 5D
    // perhaps other FTN related fields
    },
    "connectivity" : {
    "telnet" : "blah.blah.blah:1234",
    "wss": "websockets.blah.blah:4566",
    "http" : "foo.bar",
    // so on.
    }
    }

    ...the idea being structured, yet flexible, and in a common format.

    Or perhaps just an array of "connectivity" and include schemas:
    [
    "telnet://bla.blah:123",
    "wss://websockets.blah.blah",
    "https://something.other.whatever",
    "http://a.nonstandard.port:8080"
    ]



    --
    NuSkooler
    Xibalba BBS @ xibalba.l33t.codes / 44510(telnet) 44511(ssh)
    ENiGMA 1/2 BBS WHQ | Phenom | 67 | iMPURE | ACiDic
    --- ENiGMA 1/2 v0.0.12-beta (linux; x64; 12.13.1)
    * Origin: Xibalba -+- xibalba.l33t.codes:44510 (21:1/121)
  • From Avon@21:1/101 to NuSkooler on Sat Jun 6 19:06:28 2020
    On 05 Jun 2020 at 06:26p, NuSkooler pondered and said...

    Consider the following JSON (could be: XML (ew!!!), TOML, whatever):

    I think JSON is a good idea. Perhaps there may be other formats some may
    think better?? Dunno, but yeah JSON seemed to me (as a n00b) to be a way of sharing extensible data in a day that software devs could find it useful.

    I wonder also if this would be a way of feeding data using some scripts in
    and out of DNS records like Tenser and Vk3jed were discussing?

    I kinda lean towards having multiple ways to share the data.

    I'm also trying to think how such data could be added to, updated and shared
    in a decentralized way? I guess DNS sort of does that last bit, but perhaps there are other ways too?

    --- Mystic BBS v1.12 A46 2020/05/28 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From apam@21:1/126 to Avon on Sat Jun 6 17:33:53 2020
    On 05 Jun 2020 at 06:26p, NuSkooler pondered and said...

    Consider the following JSON (could be: XML (ew!!!), TOML,
    whatever):

    I think JSON is a good idea. Perhaps there may be other formats
    some may think better?? Dunno, but yeah JSON seemed to me (as a
    n00b) to be a way of sharing extensible data in a day that software
    devs could find it useful.

    I agree with JSON being a good idea :) YAML might also be a good choice
    due to it's readability and also that it supports comments.

    Have a look at: https://www.json2yaml.com/ to compare the two formats

    Andrew

    --- MagickaBBS v0.15alpha (Linux/x86_64)
    * Origin: HappyLand - telnet://magickabbs.com:2023/ (21:1/126)
  • From Gluon@21:1/151 to apam on Sat Jun 6 12:46:02 2020
    On 06 Jun 2020, apam said the following...

    I agree with JSON being a good idea :) YAML might also be a good choice due to it's readability and also that it supports comments.

    Have a look at: https://www.json2yaml.com/ to compare the two formats

    I code a lot in Python and use JSON because, well, it's everywhere. I had
    heard about YAML but never really used it. As soon as I clicked your link and saw that YAML snippet, my brain identified Python's colon and identation
    style, meaning I already knew YAML without learning it before. It's
    beautifully similar to pseudo-code, just like Python.

    ---
    Vasco aka Gluon

    --- Mystic BBS v1.12 A45 2020/02/18 (Raspberry Pi/32)
    * Origin: Geek Sphere BBS (21:1/151)
  • From tenser@21:1/101 to stizzed on Sun Jun 7 01:20:05 2020
    On 04 Jun 2020 at 11:38p, stizzed pondered and said...

    Here is the latest compilation of information we have discussed
    including in the new BBSINFO DataFile (whatever form it takes). I am posting this to further the conversation about the standard:

    BBSINFO:

    Zone // BBS ZONE NUMBER (224)
    Net // BBS NET NUMBER (10)
    Node // BBS NODE NUMBER (0)
    BbsName // BBS NAME (My New BBS)
    City // BBS Location (City)
    State // BBS Location (State)
    Country // BBS Location (Country)
    Language // BBS Language (ENG, SPN, FRC, GMN, etc)
    SysopName // SYSOP NAME (Johnny SysOp)
    Phone // BBS PHONE NUMBER (888-555-1212)
    BBSURI1 // FULL BBS URL W/PORT (telnet://bbs.domain.tld:23)
    BBSURI2 // FULL BBS URL W/PORT (rlogin://bbs.domain.tld:513)
    BBSURI3 // FULL BBS URL W/PORT (www://www.bbs.domain.tld:80)
    BBSURI4 // FULL BBS URL W/PORT (ssh://bbs.domain.tld:22)
    Nodes // TOTAL # OF NODES SUPPORTED (DIALUP & TELNET)
    Callers // AVERAGE DAILY CALLERS TO THIS BBS (25)
    Online // BBS Operating Hours-GMT (0000-0000 or 0700-1800)
    Software // BBS SOFTWARE (Mystic v1.12 a45)
    Delete // FLAG FOR DELETION BY DOWNSTREAM SYSTEMS
    Verified // DATE INFO WAS VERIFIED (20200605)
    LognType // LOGIN TYPE (1=realname/2=handle/3=choic )
    cType // BBS TYPE (1=dialup/2=telnet/3=both

    This is unstructured and basically a linear bag of fields;
    but almost all structured formats can let you group things
    together in more semantically meaningful ways. Consider
    the following, in a sort of pseudo-EBNF style grammar:
    Let a token be followed by a `?` be considered optional:
    that is, zero or one of the token. Let tokens followed
    by a `*` be considered repeated 0 or more times. Let
    tokens followed by a `+` denote 1 or more. Let
    `token1|token2` alternation, or either of those tokens.

    ftnAddress := Zone? Net Node Point? Domain?
    ftnMember := NetName ftnAddress+
    Location := City State Country
    Service := Name Host Port?
    LoginType := (RealName|Handle|Either)
    CharEncoding := (UTF-8|ASCII|ISOLATIN1|CP437)
    BBSOtherData := NumNodes? AvgCallers? AvailTimes? LoginType?
    BBS := Name SysOp Location? Language* ftnMember* (Phone|Service)+
    BBSOtherData? LastVerifiedTime CharEncoding*

    I wouldn't bake a "deleted" flag into the data format.
    Rather, if one wants to delete, that should be part of
    a "delete this entry" sort of thing.

    Note that this will map well to something like JSON or
    whatever, or even DNS. An example for my system might
    be:

    "FatDragon": {
    "name": "The Fat Dragon",
    "sysop": "Dan Cross",
    "location": { "city": "New York City" },
    "language": [ "English" ],
    "charencoding": [ "ASCII", "UTF-8" ],
    "ftn": [
    {
    "netname": "fsxnet",
    "ftnaddress": {
    "zone": 21, "net": 1, "node": 198, "domain": "fsxnet"
    }
    }
    ],
    "service": [
    { "name": "ssh", "host": "fat-dragon.org", "port": 22 },
    { "name": "http", "host": "fat-dragon.org", "port": 80 },
    { "name": "binkp", "host": "trunk.fat-dragon.org", "port": 24554 }
    ],
    "otherdata": { "numnodes": 1024 },
    "lastverified": "2020-06-06T13:13:13Z"
    }

    --- Mystic BBS v1.12 A46 2020/05/28 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From tenser@21:1/101 to Avon on Sun Jun 7 01:25:16 2020
    On 06 Jun 2020 at 07:06p, Avon pondered and said...

    I think JSON is a good idea. Perhaps there may be other formats some may think better?? Dunno, but yeah JSON seemed to me (as a n00b) to be a way of sharing extensible data in a day that software devs could find it useful.

    JSON would honestly be the best thing for this, though
    really the important part is identifying a schema; the
    actual format it's transfered in shouldn't matter all
    that much.

    I would avoid YAML: it looks deceptively simple at first,
    but it's hellaciously complex.

    --- Mystic BBS v1.12 A46 2020/05/28 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From stizzed@21:4/156 to tenser on Sat Jun 6 11:31:35 2020
    165
    On |1507 Jun 2020|08, |15tenser |08said the following...

    On 04 Jun 2020 at 11:38p, stizzed pondered and said...

    Here is the latest compilation of information we have discussed including in the new BBSINFO DataFile (whatever form it takes). I am posting this to further the conversation about the standard:

    BBSINFO:

    Zone // BBS ZONE NUMBER (224)
    Net // BBS NET NUMBER (10)
    Node // BBS NODE NUMBER (0)
    BbsName // BBS NAME (My New BBS)
    City // BBS Location (City)
    State // BBS Location (State)
    Country // BBS Location (Country)
    Language // BBS Language (ENG, SPN, FRC, GMN, etc)
    SysopName // SYSOP NAME (Johnny SysOp)
    Phone // BBS PHONE NUMBER (888-555-1212)
    BBSURI1 // FULL BBS URL W/PORT (telnet://bbs.domain.tld:23)
    BBSURI2 // FULL BBS URL W/PORT (rlogin://bbs.domain.tld:513)
    BBSURI3 // FULL BBS URL W/PORT (www://www.bbs.domain.tld:80)
    BBSURI4 // FULL BBS URL W/PORT (ssh://bbs.domain.tld:22)
    Nodes // TOTAL # OF NODES SUPPORTED (DIALUP & TELNET)
    Callers // AVERAGE DAILY CALLERS TO THIS BBS (25)
    Online // BBS Operating Hours-GMT (0000-0000 or 0700-1800)
    Software // BBS SOFTWARE (Mystic v1.12 a45)
    Delete // FLAG FOR DELETION BY DOWNSTREAM SYSTEMS
    Verified // DATE INFO WAS VERIFIED (20200605)
    LognType // LOGIN TYPE (1=realname/2=handle/3=choic )
    cType // BBS TYPE (1=dialup/2=telnet/3=both

    This is unstructured and basically a linear bag of fields;
    but almost all structured formats can let you group things
    together in more semantically meaningful ways. Consider

    Yes, yes it is. I was initially chastized for suggesting any kind of
    format. It is what it is though... :)

    the following, in a sort of pseudo-EBNF style grammar:
    Let a token be followed by a `?` be considered optional:
    that is, zero or one of the token. Let tokens followed
    by a `*` be considered repeated 0 or more times. Let
    tokens followed by a `+` denote 1 or more. Let
    `token1|token2` alternation, or either of those tokens.

    ftnAddress := Zone? Net Node Point? Domain?
    ftnMember := NetName ftnAddress+
    Location := City State Country
    Service := Name Host Port?
    LoginType := (RealName|Handle|Either)
    CharEncoding := (UTF-8|ASCII|ISOLATIN1|CP437)
    BBSOtherData := NumNodes? AvgCallers? AvailTimes? LoginType?
    BBS := Name SysOp Location? Language* ftnMember* (Phone|Service)+
    BBSOtherData? LastVerifiedTime CharEncoding*

    YES! And flat...

    I wouldn't bake a "deleted" flag into the data format.
    Rather, if one wants to delete, that should be part of
    a "delete this entry" sort of thing.

    Certainly, depending upon how the 'list' is maintained and distributed.
    As you may know, this has not been determined yet.

    Note that this will map well to something like JSON or
    whatever, or even DNS. An example for my system might
    be:

    "FatDragon": {
    "name": "The Fat Dragon",
    "sysop": "Dan Cross",
    "location": { "city": "New York City" },
    "language": [ "English" ],
    "charencoding": [ "ASCII", "UTF-8" ],
    "ftn": [
    {
    "netname": "fsxnet",
    "ftnaddress": {
    "zone": 21, "net": 1, "node": 198, "domain": "fsxnet"
    }
    }
    ],
    "service": [
    { "name": "ssh", "host": "fat-dragon.org", "port": 22 },
    { "name": "http", "host": "fat-dragon.org", "port": 80 },
    { "name": "binkp", "host": "trunk.fat-dragon.org", "port": 24554
    } ],
    "otherdata": { "numnodes": 1024 },
    "lastverified": "2020-06-06T13:13:13Z"
    }

    That looks wonderful and I like it (along with several other suggestions).

    Perhaps I'm looking at this from the wrong end? Should we be deciding upon a format now, or the information we want to capture? Or both?

    .\\ichael Batts
    a.k.a. stizzed (because, why not?)
    SysOp, The ROCK BBS III

    --- Mystic BBS v1.12 A45 2020/02/18 (Windows/32)
    * Origin: The ROCK III - therockbbs.net - TELNET:10023 (21:4/156)
  • From tenser@21:1/101 to stizzed on Sun Jun 7 05:43:30 2020
    On 06 Jun 2020 at 11:31a, stizzed pondered and said...

    Yes, yes it is. I was initially chastized for suggesting any kind of format. It is what it is though... :)

    Well, there's a difference between a concrete format
    like a Pascal record, and a structured schema. What
    I tried to present is the latter; how you serialize
    that into any particular format is an implementation
    detail.

    the following, in a sort of pseudo-EBNF style grammar:
    Let a token be followed by a `?` be considered optional:
    that is, zero or one of the token. Let tokens followed
    by a `*` be considered repeated 0 or more times. Let
    tokens followed by a `+` denote 1 or more. Let
    `token1|token2` alternation, or either of those tokens.

    ftnAddress := Zone? Net Node Point? Domain?
    ftnMember := NetName ftnAddress+
    Location := City State Country
    Service := Name Host Port?
    LoginType := (RealName|Handle|Either)
    CharEncoding := (UTF-8|ASCII|ISOLATIN1|CP437)
    BBSOtherData := NumNodes? AvgCallers? AvailTimes? LoginType?
    BBS := Name SysOp Location? Language* ftnMember* (Phone|Service)+
    BBSOtherData? LastVerifiedTime CharEncoding*

    YES! And flat...

    Not flat, no; notice how the different data nest
    within each other. The point is that this describes
    the _structure_ of the data, but is independent of
    how one _formats_ that structure.

    I wouldn't bake a "deleted" flag into the data format.
    Rather, if one wants to delete, that should be part of
    a "delete this entry" sort of thing.

    Certainly, depending upon how the 'list' is maintained and distributed. As you may know, this has not been determined yet.

    Here's where you want to draw a strong distinction
    between the data and how it's structured and operations
    on that data.

    Perhaps I'm looking at this from the wrong end? Should we be deciding upon a format now, or the information we want to capture? Or both?

    Fundamentally, you want to describe the data, but in a
    structured and semantically meaningful way. The exact
    serialization of that data into any particular format
    is a secondary consideration.

    That said, I would strongly recommend JSON. It's
    lightweight, and it's available pretty much everywhere.
    It does basically everything you need and can be
    upwards compatible if you add things to the schema.

    I would strongly recommend _against_ an ersatz format,
    or an unstructured line-based thing a la the current
    nodelist format.

    --- Mystic BBS v1.12 A46 2020/05/28 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From Avon@21:1/101 to tenser on Sun Jun 7 16:44:59 2020
    On 07 Jun 2020 at 05:43a, tenser pondered and said...

    Fundamentally, you want to describe the data, but in a
    structured and semantically meaningful way. The exact
    serialization of that data into any particular format
    is a secondary consideration.

    That said, I would strongly recommend JSON. It's
    lightweight, and it's available pretty much everywhere.
    It does basically everything you need and can be
    upwards compatible if you add things to the schema.

    I would strongly recommend _against_ an ersatz format,
    or an unstructured line-based thing a la the current
    nodelist format.

    I've found myself falling down the rabbit hole of wanting to look at how the data could / should be most usefully formatted first rather (e.g. JSON)
    rather than focus on what data would be good to allow for and it's structure and semantics. This is not an area of experience I've had much to do with.
    But I do take the point about trying to nail down the what before you look at the how. I hope I have understood this difference correctly.

    ftnAddress := Zone? Net Node Point? Domain?
    ftnMember := NetName ftnAddress+
    Location := City State Country
    Service := Name Host Port?
    LoginType := (RealName|Handle|Either)
    CharEncoding := (UTF-8|ASCII|ISOLATIN1|CP437)
    BBSOtherData := NumNodes? AvgCallers? AvailTimes? LoginType?
    BBS := Name SysOp Location? Language* ftnMember* (Phone|Service)
    BBSOtherData? LastVerifiedTime CharEncoding*

    YES! And flat...

    Not flat, no; notice how the different data nest
    within each other. The point is that this describes
    the _structure_ of the data, but is independent of
    how one _formats_ that structure.

    What I'm trying to grapple with is how do we know when we think we have captured enough possible data fields of interest to then be able to move on
    to creating ways to store, share and modify it?

    I'm also not clear if creating these data fields is a one shot deal or will what we use to store it all in and/or build other things from using the data, allow for new fields to be added e.g. let's say we 6 months later wanted to
    add a co-sysop field just as an example..

    --- Mystic BBS v1.12 A46 2020/05/28 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From tenser@21:1/101 to Avon on Mon Jun 8 00:45:25 2020
    On 07 Jun 2020 at 04:44p, Avon pondered and said...

    I've found myself falling down the rabbit hole of wanting to look at how the data could / should be most usefully formatted first rather (e.g. JSON) rather than focus on what data would be good to allow for and it's structure and semantics. This is not an area of experience I've had much to do with. But I do take the point about trying to nail down the what before you look at the how. I hope I have understood this difference correctly.

    That's fair. It's useful to be able to look at examples
    to see if all the useful things are being captured.

    What vs how is a great way to think about it.

    What I'm trying to grapple with is how do we know when we think we have captured enough possible data fields of interest to then be able to move on to creating ways to store, share and modify it?

    In short, you don't. The trick is to structure the data
    in such a way that's it is extensible in a backwards
    compatible way. That is, adding a field shouldn't break
    existing software. Formats like JSON are great for
    applications like that since each datum is labeled with
    a key; things like CSV or other unstructured line-based
    formats are terrible since inserting a field between
    existing fields breaks all existing software.

    I'm also not clear if creating these data fields is a one shot deal or will what we use to store it all in and/or build other things from using the data, allow for new fields to be added e.g. let's say we 6 months later wanted to add a co-sysop field just as an example..

    Surely you'll want to add things over time. New services
    come, old ones will fall out of use, etc. But by using
    a structured data format with semantically meaningful tagging
    and by having a well-defined, extensible schema, you can
    accommodate those changes as they occur.

    --- Mystic BBS v1.12 A46 2020/05/28 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)
  • From rushfan@21:1/101 to tenser on Mon Jun 15 11:04:20 2020
    compatible way. That is, adding a field shouldn't break
    existing software. Formats like JSON are great for
    applications like that since each datum is labeled with
    a key; things like CSV or other unstructured line-based
    formats are terrible since inserting a field between
    existing fields breaks all existing software.

    It's pretty much guaranteed that whatever is needed today won't be the full
    set of data needed at some point in the future. The examples in JSON listed all seem super reasonable as starting points. I really like breaking out the URL types by connection type. Should be easy to also allow it to integrate into bbs lists and also dialing directories.

    FWIW both WWIV and Synchronet used JSON for the bbslist. In WWIV we keep switching more "pascal record" type files to JSON (and still writing out the legacy binary format for compatibility). It provides real easy extensibility since you can use keys vs position to evolve the format over time if/as
    needed.

    -rushfan

    --- Mystic BBS v1.12 A46 2020/05/28 (Windows/32)
    * Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (21:1/101)