Docs: Mandocs
This commit is contained in:
Agie Ashwood
parent
b073b06e03
commit
161944eab7
|
|
@ -7,3 +7,5 @@ rm -rf docs.tmp
|
|||
mv docs/index.md docs/readme.md
|
||||
sed -i '1s;^;\n\n;' docs/readme.md
|
||||
sed -i '1s;^;\n\n;' docs/Daisy/Daisy.md
|
||||
echo $'\n' >> docs/readme.md
|
||||
cat src/readme_append.md >> docs/readme.md
|
||||
|
|
|
|||
|
|
@ -180,3 +180,88 @@ sphinx-quickstart on Fri Jul 26 23:30:55 2024. -->
|
|||
* [`Server.app`](/PierMesh/piermesh/src/branch/main/docs/Splash/serve.md#Splash.serve.Server.app)
|
||||
* [`Server.catch`](/PierMesh/piermesh/src/branch/main/docs/Splash/serve.md#Splash.serve.Server.catch)
|
||||
* [`Server.sendToPeer()`](/PierMesh/piermesh/src/branch/main/docs/Splash/serve.md#Splash.serve.Server.sendToPeer)
|
||||
|
||||
|
||||
# System Overview
|
||||
|
||||
PierMesh has two main events loops to learn about: the TUI and the service.
|
||||
|
||||
## TUI
|
||||
|
||||
[🔗 Docs](https://git.utopic.work/PierMesh/piermesh/src/branch/main/docs/ui.md)
|
||||
|
||||
[🔗 Source](https://git.utopic.work/PierMesh/piermesh/src/branch/main/src/ui.py)
|
||||
|
||||
The TUI is provided via [Textual's](https://textual.textualize.io/) library. It's a relatively simple application that gives us a quick overview of system statistics in the way of memory and cpu usage as well as scrollable logs. You can toggle full screen logs with f and close the TUI and service with q.
|
||||
|
||||
## The Service
|
||||
|
||||
PierMesh runs a number of loops under the hood. These are primarily initialized in the main loop of run with a special logging loop outside of that.
|
||||
|
||||
Note that we make heavy use of [Meshtastic's Python API](https://github.com/meshtastic/python).
|
||||
|
||||
### run
|
||||
|
||||
[🔗 run.py docs](https://git.utopic.work/PierMesh/piermesh/src/branch/main/docs/run.md)
|
||||
|
||||
[🔗 run.py source](https://git.utopic.work/PierMesh/piermesh/src/branch/main/src/run.py)
|
||||
|
||||
#### run.main
|
||||
|
||||
In run.main we (in order)
|
||||
1. Initialize the `Node` class to a global `nodeOb`.
|
||||
2. Wait for x seconds determined by command line arguments (see the falin and marcille scripts in scripts)
|
||||
3. Initialize our `Transceiver` class (and the corresponding hardware)
|
||||
4. Initialize our `Server` class
|
||||
5. Create an async task running the `Node.spongeListen` method which looks for updates from the filter system
|
||||
6. Kick the loop on with `await asyncio.sleep(1)` (we do this to kick on all async loops so I will omit this step going forward)
|
||||
7. Create an async task running the `Transceiver.checkProgress` method which checks for packet reception progress and resends packets if necessary
|
||||
8. Create an async task running `Node.monitor` which checks and reports system usage
|
||||
9. Create an async task running the `Transceiver.announce` method which broadcasts a `Packets.Message` containing network mapping information
|
||||
10. Last we start the `Microdot` server loop
|
||||
|
||||
### ..
|
||||
Travelling out of the run.main thread to the primary ____main____ code we see the other two threads: one running the `logPassLoop` loop which is created in the same way as the main thread: `lplThread = threading.Thread(target=asyncio.run, args=(logPassLoop(),))` and one running the TUI loop which has it's own system we just kick on by instantiating the TUI class and calling .run() on.
|
||||
|
||||
## Packet lifecycle
|
||||
|
||||
Prior to this we've been talking relatively high level but if you've read this far you probably want more low level details. Let's talk about packets.
|
||||
|
||||
### Packets.Packet
|
||||
|
||||
This is a base singular packet. At the very least a packet will contain: a packets ID which determines what set of packets (message) a singular packet belongs to if any, a packetNumber which determines which order the data will be put in, a packetCount which is, well a count of all the packets in a set of packets and in most cases data which is an lzma compressed msgpack encoded chunk of data to be fused together when the full set of packets is received.
|
||||
|
||||
### Packets.HeaderPacket
|
||||
|
||||
This is the metadata packet for a set of packets (message) which handles the details necessary for routing and action triggering
|
||||
|
||||
### Packets.Message
|
||||
|
||||
A set of packets instantiated primarily from lzma compressed msgpack encoded bytes
|
||||
|
||||
### You keep saying msgpack what is that?
|
||||
|
||||
[msgpack](https://msgpack.org/) is what I landed on for bundling metadata and arbitrary bytes after initially using bson as msgpack is leaner, simpler and better adopted for cross platform data interchange
|
||||
|
||||
### Transmitting
|
||||
|
||||
Once we have our msgpack encoded lzma compressed data we can send it with `Transceiver.addPackets` this method creates a `Message` and sends each `Packet` (dumped to binary data) over LoRa using Meshtastic's Python interface via the `Transceiver.send` method.
|
||||
|
||||
### Receiving
|
||||
|
||||
On the other end when we receive a `Packet` we pass it directly into `Sponge.base.sieve` which checks: if it's msgpack encoded and if it's a packet sent by the same node. In both cases it skips the packet, otherwise we sieve!
|
||||
|
||||
### Sieving
|
||||
|
||||
Primarily in this stage we are gathering packets and adding them to an object containing all of the packet info and data we've received for that `Message`.
|
||||
|
||||
Once a `Message` is completed we pass it on to the appropriate protocol from `Sponge.Protocols`. These will pass the `Message` wherever it needs to go from there which can be any number of places.
|
||||
|
||||
To avoid unnecessary manual documentation labor for future devs I will not list everything that could happen here but here are some examples:
|
||||
- Updating a Catch
|
||||
- Sending a message up to a peer on the node
|
||||
- Sending a Diffie Hellman handshake
|
||||
|
||||
# Fin
|
||||
|
||||
Well not really, there's more to learn but these are the basics. There's more in the docs above and you can always submit an issue or reach me at info@piermesh.net or on Tumblr at utopicwork.
|
||||
|
|
@ -6,7 +6,15 @@ This is the monorepo for PierMesh.
|
|||
|
||||
# Docs
|
||||
|
||||
<<<<<<< HEAD
|
||||
<<<<<<< HEAD
|
||||
You can find the full docs here: [docs/](https://git.utopic.work/PierMesh/piermesh/src/branch/main/docs)
|
||||
=======
|
||||
You can find the full docs here: [docs/](docs/)
|
||||
>>>>>>> 3ad19a9 (Docs: Clean up primary readme)
|
||||
=======
|
||||
You can find the full docs here: [docs/](https://git.utopic.work/PierMesh/piermesh/src/branch/main/docs)
|
||||
>>>>>>> 6f5bfe4 (Docs link fix)
|
||||
|
||||
# How to use
|
||||
|
||||
|
|
|
|||
|
|
@ -257,6 +257,7 @@ class DHEFern:
|
|||
"""
|
||||
return os.urandom(16)
|
||||
|
||||
# TODO: Build in transport security (node/node)
|
||||
def encrypt(self, data, nodeID: str, isDict: bool = True):
|
||||
"""
|
||||
Do Fernet encryption
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ class Message:
|
|||
senderDisplayName: int,
|
||||
recipient: int,
|
||||
recipientNode: int,
|
||||
cryptographyInfo,
|
||||
dataSize: int = 128,
|
||||
wantFullResponse: bool = False,
|
||||
packetsClass: int = 0,
|
||||
|
|
@ -71,7 +72,10 @@ class Message:
|
|||
)
|
||||
self.packets = packets
|
||||
else:
|
||||
bytesObject = lzma.compress(bytesObject)
|
||||
# Data passed in by peers should already have been e2ee encrypted by SubtleCrypto
|
||||
# Transport encryption
|
||||
# bytesObject = lzma.compress(bytesObject, str(recipientNode).zfill(6), isDict=False)
|
||||
bytesObject = cryptographyInfo.encrypt(bytesObject, self.no)
|
||||
packets = []
|
||||
self.packetsID = random.randrange(0, 999999)
|
||||
pnum = 1
|
||||
|
|
|
|||
|
|
@ -227,6 +227,7 @@ class Transceiver:
|
|||
senderName,
|
||||
recipient,
|
||||
recipientNode,
|
||||
self.cryptographyInfo,
|
||||
packetsClass=packetsClass,
|
||||
)
|
||||
for p in tp.packets:
|
||||
|
|
|
|||
|
|
@ -0,0 +1,2 @@
|
|||
cloc --exclude-dir=Splash,stale .
|
||||
cloc Splash/serve.py
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
# System Overview
|
||||
|
||||
PierMesh has two main events loops to learn about: the TUI and the service.
|
||||
|
||||
## TUI
|
||||
|
||||
[🔗 Docs](https://git.utopic.work/PierMesh/piermesh/src/branch/main/docs/ui.md)
|
||||
|
||||
[🔗 Source](https://git.utopic.work/PierMesh/piermesh/src/branch/main/src/ui.py)
|
||||
|
||||
The TUI is provided via [Textual's](https://textual.textualize.io/) library. It's a relatively simple application that gives us a quick overview of system statistics in the way of memory and cpu usage as well as scrollable logs. You can toggle full screen logs with f and close the TUI and service with q.
|
||||
|
||||
## The Service
|
||||
|
||||
PierMesh runs a number of loops under the hood. These are primarily initialized in the main loop of run with a special logging loop outside of that.
|
||||
|
||||
Note that we make heavy use of [Meshtastic's Python API](https://github.com/meshtastic/python).
|
||||
|
||||
### run
|
||||
|
||||
[🔗 run.py docs](https://git.utopic.work/PierMesh/piermesh/src/branch/main/docs/run.md)
|
||||
|
||||
[🔗 run.py source](https://git.utopic.work/PierMesh/piermesh/src/branch/main/src/run.py)
|
||||
|
||||
#### run.main
|
||||
|
||||
In run.main we (in order)
|
||||
1. Initialize the `Node` class to a global `nodeOb`.
|
||||
2. Wait for x seconds determined by command line arguments (see the falin and marcille scripts in scripts)
|
||||
3. Initialize our `Transceiver` class (and the corresponding hardware)
|
||||
4. Initialize our `Server` class
|
||||
5. Create an async task running the `Node.spongeListen` method which looks for updates from the filter system
|
||||
6. Kick the loop on with `await asyncio.sleep(1)` (we do this to kick on all async loops so I will omit this step going forward)
|
||||
7. Create an async task running the `Transceiver.checkProgress` method which checks for packet reception progress and resends packets if necessary
|
||||
8. Create an async task running `Node.monitor` which checks and reports system usage
|
||||
9. Create an async task running the `Transceiver.announce` method which broadcasts a `Packets.Message` containing network mapping information
|
||||
10. Last we start the `Microdot` server loop
|
||||
|
||||
### ..
|
||||
Travelling out of the run.main thread to the primary ____main____ code we see the other two threads: one running the `logPassLoop` loop which is created in the same way as the main thread: `lplThread = threading.Thread(target=asyncio.run, args=(logPassLoop(),))` and one running the TUI loop which has it's own system we just kick on by instantiating the TUI class and calling .run() on.
|
||||
|
||||
## Packet lifecycle
|
||||
|
||||
Prior to this we've been talking relatively high level but if you've read this far you probably want more low level details. Let's talk about packets.
|
||||
|
||||
### Packets.Packet
|
||||
|
||||
This is a base singular packet. At the very least a packet will contain: a packets ID which determines what set of packets (message) a singular packet belongs to if any, a packetNumber which determines which order the data will be put in, a packetCount which is, well a count of all the packets in a set of packets and in most cases data which is an lzma compressed msgpack encoded chunk of data to be fused together when the full set of packets is received.
|
||||
|
||||
### Packets.HeaderPacket
|
||||
|
||||
This is the metadata packet for a set of packets (message) which handles the details necessary for routing and action triggering
|
||||
|
||||
### Packets.Message
|
||||
|
||||
A set of packets instantiated primarily from lzma compressed msgpack encoded bytes
|
||||
|
||||
### You keep saying msgpack what is that?
|
||||
|
||||
[msgpack](https://msgpack.org/) is what I landed on for bundling metadata and arbitrary bytes after initially using bson as msgpack is leaner, simpler and better adopted for cross platform data interchange
|
||||
|
||||
### Transmitting
|
||||
|
||||
Once we have our msgpack encoded lzma compressed data we can send it with `Transceiver.addPackets` this method creates a `Message` and sends each `Packet` (dumped to binary data) over LoRa using Meshtastic's Python interface via the `Transceiver.send` method.
|
||||
|
||||
### Receiving
|
||||
|
||||
On the other end when we receive a `Packet` we pass it directly into `Sponge.base.sieve` which checks: if it's msgpack encoded and if it's a packet sent by the same node. In both cases it skips the packet, otherwise we sieve!
|
||||
|
||||
### Sieving
|
||||
|
||||
Primarily in this stage we are gathering packets and adding them to an object containing all of the packet info and data we've received for that `Message`.
|
||||
|
||||
Once a `Message` is completed we pass it on to the appropriate protocol from `Sponge.Protocols`. These will pass the `Message` wherever it needs to go from there which can be any number of places.
|
||||
|
||||
To avoid unnecessary manual documentation labor for future devs I will not list everything that could happen here but here are some examples:
|
||||
- Updating a Catch
|
||||
- Sending a message up to a peer on the node
|
||||
- Sending a Diffie Hellman handshake
|
||||
|
||||
# Fin
|
||||
|
||||
Well not really, there's more to learn but these are the basics. There's more in the docs above and you can always submit an issue or reach me at info@piermesh.net or on Tumblr at utopicwork.
|
||||
Loading…
Reference in New Issue