This commit is contained in:
Agie Ashwood 2024-07-29 03:47:27 +00:00
parent bd25941332
commit 0887586a93
23 changed files with 142 additions and 143 deletions

View File

@ -1,17 +0,0 @@
Primary filtering functionality
Dispatches to Protocols
===============================
### *class* Filters.base.Filter(cache, onodeID, todo, cLog)
#### *async* protoRoute(completeMessage)
Shorthand reference
### *class* Filters.Protocols.bubble.filter(completeMessage, recipient, recipientNode, onodeID, todo)
### *class* Filters.Protocols.catch.filter(completeMessage, recipient, recipientNode, todo)
### *class* Filters.Protocols.cryptography.filter(completeMessage, recipientNode, todo)
### *class* Filters.Protocols.map.filter(completeMessage, todo)

View File

@ -1 +0,0 @@
# Configuration utilities

View File

@ -50,12 +50,8 @@ Add edge between peers on inner map
Add peer on inner map
#### id
PierMesh node ID
* **Type:**
int
* **Parameters:**
**id** (*int*) PierMesh node ID
#### addoe(nodea: int, nodeb: int)
@ -65,12 +61,8 @@ Add edge between nodes on outer map
Add node on outer map
#### id
PierMesh node ID
* **Type:**
int
* **Parameters:**
**id** (*int*) PierMesh node ID
#### doLookup(onodeID: str)
@ -97,4 +89,4 @@ Import map from path
#### render(pathPrefix: str = '')
Render outer and inner network mao to disk at the given path prefix
Render outer and inner network map to disk at the given path prefix

View File

@ -1,3 +0,0 @@
# Data routing logic and data
### *class* Siph.router.Router(cLog, nfpath='server.info')

17
docs/Sponge/base.md Normal file
View File

@ -0,0 +1,17 @@
Primary filtering functionality
Dispatches to Protocols
===============================
### *class* Sponge.base.Filter(cache, onodeID, todo, cLog)
#### *async* protoRoute(completeMessage)
Shorthand reference
### *class* Sponge.Protocols.bubble.filter(completeMessage, recipient, recipientNode, onodeID, todo)
### *class* Sponge.Protocols.catch.filter(completeMessage, recipient, recipientNode, todo)
### *class* Sponge.Protocols.cryptography.filter(completeMessage, recipientNode, todo)
### *class* Sponge.Protocols.map.filter(completeMessage, todo)

View File

@ -15,8 +15,8 @@ sphinx-quickstart on Fri Jul 26 23:30:55 2024. -->
* [`Node.action_sendCatch()`](/PierMesh/piermesh/src/branch/main/docs/run.md#run.Node.action_sendCatch)
* [`Node.action_sendToPeer()`](/PierMesh/piermesh/src/branch/main/docs/run.md#run.Node.action_sendToPeer)
* [`Node.cLog()`](/PierMesh/piermesh/src/branch/main/docs/run.md#run.Node.cLog)
* [`Node.fListen()`](/PierMesh/piermesh/src/branch/main/docs/run.md#run.Node.fListen)
* [`Node.monitor()`](/PierMesh/piermesh/src/branch/main/docs/run.md#run.Node.monitor)
* [`Node.spongeListen()`](/PierMesh/piermesh/src/branch/main/docs/run.md#run.Node.spongeListen)
* [`Node.toLog`](/PierMesh/piermesh/src/branch/main/docs/run.md#run.Node.toLog)
* [TUI application](/PierMesh/piermesh/src/branch/main/docs/ui.md)
* [`TUI`](/PierMesh/piermesh/src/branch/main/docs/ui.md#ui.TUI)
@ -31,7 +31,6 @@ sphinx-quickstart on Fri Jul 26 23:30:55 2024. -->
* [`TUI.do_set_mem()`](/PierMesh/piermesh/src/branch/main/docs/ui.md#ui.TUI.do_set_mem)
* [`TUI.do_write_line()`](/PierMesh/piermesh/src/branch/main/docs/ui.md#ui.TUI.do_write_line)
* [`TUI.on_mount()`](/PierMesh/piermesh/src/branch/main/docs/ui.md#ui.TUI.on_mount)
* [Configuration utilities](/PierMesh/piermesh/src/branch/main/docs/Siph/config.md)
* [Network map representation](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md)
* [`Network`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network)
* [`Network.omap`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.omap)
@ -41,18 +40,14 @@ sphinx-quickstart on Fri Jul 26 23:30:55 2024. -->
* [`Network.addLookup()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.addLookup)
* [`Network.addie()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.addie)
* [`Network.addin()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.addin)
* [`Network.id`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.id)
* [`Network.addoe()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.addoe)
* [`Network.addon()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.addon)
* [`Network.id`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#id0)
* [`Network.doLookup()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.doLookup)
* [`Network.export()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.export)
* [`Network.path`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.path)
* [`Network.getRoute()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.getRoute)
* [`Network.mimport()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.mimport)
* [`Network.render()`](/PierMesh/piermesh/src/branch/main/docs/Siph/map.md#Siph.map.Network.render)
* [Data routing logic and data](/PierMesh/piermesh/src/branch/main/docs/Siph/router.md)
* [`Router`](/PierMesh/piermesh/src/branch/main/docs/Siph/router.md#Siph.router.Router)
* [Schemaless binary database](/PierMesh/piermesh/src/branch/main/docs/Components/daisy.md)
* [`Daisy`](/PierMesh/piermesh/src/branch/main/docs/Components/daisy.md#Components.daisy.Daisy)
* [`Daisy.get()`](/PierMesh/piermesh/src/branch/main/docs/Components/daisy.md#Components.daisy.Daisy.get)
@ -71,12 +66,12 @@ sphinx-quickstart on Fri Jul 26 23:30:55 2024. -->
* [`get()`](/PierMesh/piermesh/src/branch/main/docs/Components/hopper.md#Components.hopper.get)
* [`post()`](/PierMesh/piermesh/src/branch/main/docs/Components/hopper.md#Components.hopper.post)
* [`DHEFern`](/PierMesh/piermesh/src/branch/main/docs/Cryptography/DHEFern.md)
* [`Filter`](/PierMesh/piermesh/src/branch/main/docs/Filters/base.md)
* [`Filter.protoRoute()`](/PierMesh/piermesh/src/branch/main/docs/Filters/base.md#Filters.base.Filter.protoRoute)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Filters/base.md#Filters.Protocols.bubble.filter)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Filters/base.md#Filters.Protocols.catch.filter)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Filters/base.md#Filters.Protocols.cryptography.filter)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Filters/base.md#Filters.Protocols.map.filter)
* [`Filter`](/PierMesh/piermesh/src/branch/main/docs/Sponge/base.md)
* [`Filter.protoRoute()`](/PierMesh/piermesh/src/branch/main/docs/Sponge/base.md#Sponge.base.Filter.protoRoute)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Sponge/base.md#Sponge.Protocols.bubble.filter)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Sponge/base.md#Sponge.Protocols.catch.filter)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Sponge/base.md#Sponge.Protocols.cryptography.filter)
* [`filter`](/PierMesh/piermesh/src/branch/main/docs/Sponge/base.md#Sponge.Protocols.map.filter)
* [Header packet: Metadata packet](/PierMesh/piermesh/src/branch/main/docs/Packets/HeaderPacket.md)
* [`Header`](/PierMesh/piermesh/src/branch/main/docs/Packets/HeaderPacket.md#Packets.HeaderPacket.Header)
* [Base packet](/PierMesh/piermesh/src/branch/main/docs/Packets/Packet.md)

View File

@ -25,7 +25,7 @@ Derive key via diffie hellman key exchange
Map new network data to internal network map
#### SEE ALSO
`Bubble.network.Network`
`Siph.network.Network`
: Layered graph etwork representation
#### *async* action_sendCatch(data: dict)
@ -63,7 +63,11 @@ Convenience function that logs to the ui and log files
* **Return type:**
None
#### *async* fListen()
#### *async* monitor()
Monitor and log ram and cpu usage
#### *async* spongeListen()
Loop to watch for tasks to do
@ -75,10 +79,6 @@ Loop to watch for tasks to do
We use a common technique here that calls the function from our preloaded actions via dictionary entry
#### *async* monitor()
Monitor and log ram and cpu usage
#### toLog
We store logs to be processed here

View File

@ -1,4 +1,5 @@
import base64, os
import base64
import os
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import dh

View File

@ -1,18 +0,0 @@
Primary filtering functionality
Dispatches to Protocols
===============================
.. autoclass:: Filters.base.Filter
:members:
.. autoclass:: Filters.Protocols.bubble.filter
:members:
.. autoclass:: Filters.Protocols.catch.filter
:members:
.. autoclass:: Filters.Protocols.cryptography.filter
:members:
.. autoclass:: Filters.Protocols.map.filter
:members:

View File

View File

@ -1,3 +0,0 @@
Configuration utilities
==========================

View File

@ -99,7 +99,7 @@ class Network:
"""
Add node on outer map
Attributes
Parameters
----------
id: int
PierMesh node ID
@ -116,7 +116,7 @@ class Network:
"""
Add peer on inner map
Attributes
Parameters
----------
id: int
PierMesh node ID
@ -137,7 +137,7 @@ class Network:
def render(self, pathPrefix: str = ""):
"""
Render outer and inner network mao to disk at the given path prefix
Render outer and inner network map to disk at the given path prefix
"""
nx.draw(self.omap)
plt.savefig(pathPrefix + "omap.png")

View File

@ -1,5 +0,0 @@
Data routing logic and data
===========================
.. autoclass:: Siph.router.Router
:members:

View File

@ -1,26 +1,20 @@
from Components.daisy import Cache
from Packets.Packets import Packets
import msgpack, lzma
from Packets.Packet import Packet
from Packets.Packets import Packets
import logging
import Filters.Protocols.bubble
import Filters.Protocols.map
import Filters.Protocols.catch
import Filters.Protocols.cryptography
import asyncio
import msgpack
import traceback
# ✅ TODO: Cache integration for messages
from Packets.Packets import Packets
import Sponge.Protocols.bubble
import Sponge.Protocols.map
import Sponge.Protocols.catch
import Sponge.Protocols.cryptography
class Filter:
def __init__(self, cache, onodeID, todo, cLog):
self.cLog = cLog
self.cache = cache
# Note: Messages is temporary storage
# for unfinished messages
"""
Messages is temporary storage for unfinished messages
"""
self.messages = {}
self.completed = []
self.todo = todo
@ -58,15 +52,15 @@ class Filter:
protocol = await self.protoMap(m["packetsClass"])
self.cLog(20, "Protocol: " + protocol)
if protocol == "bubble":
await Filters.Protocols.bubble.filter(
await Sponge.Protocols.bubble.filter(
m, recipient, recipientNode, self.onodeID, self.todo
)
elif protocol == "map":
await Filters.Protocols.map.filter(m, self.todo)
await Sponge.Protocols.map.filter(m, self.todo)
elif protocol == "catch":
await Filters.Protocols.catch.filter(m, recipient, recipientNode, self.todo)
await Sponge.Protocols.catch.filter(m, recipient, recipientNode, self.todo)
elif protocol == "cryptography":
await Filters.Protocols.cryptography.filter(
await Sponge.Protocols.cryptography.filter(
completeMessage, recipientNode, self.todo
)
else:

18
src/Sponge/base.rst Normal file
View File

@ -0,0 +1,18 @@
Primary filtering functionality
Dispatches to Protocols
===============================
.. autoclass:: Sponge.base.Filter
:members:
.. autoclass:: Sponge.Protocols.bubble.filter
:members:
.. autoclass:: Sponge.Protocols.catch.filter
:members:
.. autoclass:: Sponge.Protocols.cryptography.filter
:members:
.. autoclass:: Sponge.Protocols.map.filter
:members:

View File

@ -13,7 +13,7 @@ PierMesh documentation
../Siph/*
../Components/*
../Cryptography/*
../Filters/*
../Sponge/*
../Packets/*
../Transmission/*
../webui/*

View File

@ -1,19 +1,25 @@
from meshtastic import logging, os
from Filters.base import Filter
from Siph.router import Router
# PierMesh libraries
from Sponge.base import Filter
from Siph.map import Network
from Components.daisy import Catch
from Components.daisy import Cache
from webui.serve import Server
from Transmission.transmission import Transmitter
from Cryptography.DHEFern import DHEFern
from ui import TUI
import logging
import os
import asyncio
import sys
import time
import psutil
import logging
import datetime
from Cryptography.DHEFern import DHEFern
from microdot import Request
import traceback
from ui import TUI
import threading, os
import threading
import random
from microdot import Request
if __name__ == "__main__":
@ -61,17 +67,23 @@ class Node:
for a in actionsList:
self.actions[a.split("_")[1]] = getattr(self, a)
self.cLog(20, "Past action mapping")
self.r = Router(self.cLog, nfpath=serverInfoFile)
self.cLog(20, "Router initialized")
self.onodeID = str(self.r.serverInfo.get()["nodeID"])
self.catch = self.r.c
self.cache = self.r.cache
self.s = None
self.network = Network()
self.catch = Catch(walk=True)
self.cache = Cache(walk=True)
self.serverInfo = self.cache.get(serverInfoFile)
if self.serverInfo == False:
self.cache.create(serverInfoFile, {"nodeID": random.randrange(0, 1000000)})
self.serverInfo = self.cache.get(serverInfoFile)
self.network.addin(self.serverInfo.get()["nodeID"])
self.cLog(20, "Siph network stack initialized")
self.onodeID = str(self.serverInfo.get()["nodeID"])
self.server = None
self.todo = []
self.f = Filter(self.cache, self.onodeID, self.todo, self.cLog)
self.sponge = Filter(self.cache, self.onodeID, self.todo, self.cLog)
self.cLog(20, "Filter initialized")
# self.cLog(30, sys.argv)
self.t = None
self.oTransmitter = None
self.cLog(20, "Cryptography initializing")
self.cryptographyInfo = DHEFern(self.cache, nodeNickname, self.cLog)
self.cLog(20, "Cryptography initialized")
@ -123,7 +135,7 @@ class Node:
tuiOb.do_set_cpu_percent(float(cpup))
tuiOb.do_set_mem(memmb)
async def fListen(self):
async def spongeListen(self):
"""
Loop to watch for tasks to do
@ -161,7 +173,7 @@ class Node:
webui.serve.Server.sendToPeer: Function to actually execute the action
"""
self.s.sendToPeer(data["recipient"], data["res"])
self.server.sendToPeer(data["recipient"], data["res"])
async def action_sendCatch(self, data: dict):
"""
@ -171,8 +183,8 @@ class Node:
--------
Bubble.router.Router: Routing class
"""
res = self.r.getCatch(data["head"], data["body"], fins=data["fins"])
self.s.sendToPeer(data["recipient"], res)
res = self.catch.get(data["head"], data["body"], fins=data["fins"])
self.server.sendToPeer(data["recipient"], res)
async def action_map(self, data: dict):
"""
@ -180,11 +192,11 @@ class Node:
See Also
--------
Bubble.network.Network: Layered graph etwork representation
Siph.network.Network: Layered graph etwork representation
"""
self.r.n.addLookup(data["onodeID"], data["mnodeID"])
self.network.addLookup(data["onodeID"], data["mnodeID"])
self.cLog(20, "Lookup addition done")
self.r.n.addon(data["onodeID"])
self.network.addon(data["onodeID"])
async def action_initNodeDH(self, data: dict):
"""
@ -195,7 +207,7 @@ class Node:
Cryptography.DHEFern.DHEFern: End to end encryption functionality
"""
if self.cryptographyInfo.getRecord("key", data["onodeID"]) == False:
await self.t.initNodeDH(
await self.oTransmitter.initNodeDH(
self.cryptographyInfo, int(data["mnodeID"]), data["onodeID"]
)
@ -220,7 +232,7 @@ async def logPassLoop():
See Also
--------
tui.TUI: TUI implementation
ui.TUI: TUI implementation
"""
global tuiOb, nodeOb
while True:
@ -243,34 +255,45 @@ async def main():
"""
global nodeOb
try:
n = Node()
nodeOb = n
nodeOb = Node()
nodeOb.cLog(20, "Starting up")
nodeOb.cLog(20, "Staggering {0} seconds, please wait".format(sys.argv[4]))
time.sleep(int(sys.argv[4]))
n.t = Transmitter(
sys.argv[1], n.f, n.onodeID, n.cache, n.catch, n.cryptographyInfo, n.cLog
nodeOb.oTransmitter = Transmitter(
sys.argv[1],
nodeOb.sponge,
nodeOb.onodeID,
nodeOb.cache,
nodeOb.catch,
nodeOb.cryptographyInfo,
nodeOb.cLog,
)
n.s = Server(n.t, n.catch, n.onodeID, n.r.n, n.cLog)
n.mTasks["list"] = asyncio.create_task(n.fListen())
nodeOb.server = Server(
nodeOb.oTransmitter,
nodeOb.catch,
nodeOb.onodeID,
nodeOb.network,
nodeOb.cLog,
)
nodeOb.mTasks["list"] = asyncio.create_task(nodeOb.spongeListen())
await asyncio.sleep(1)
n.mTasks["pct"] = asyncio.create_task(n.t.progressCheck())
nodeOb.mTasks["pct"] = asyncio.create_task(nodeOb.oTransmitter.progressCheck())
await asyncio.sleep(1)
n.mTasks["mon"] = asyncio.create_task(n.monitor())
nodeOb.mTasks["mon"] = asyncio.create_task(nodeOb.monitor())
await asyncio.sleep(1)
n.mTasks["announce"] = asyncio.create_task(n.t.announce())
nodeOb.mTasks["announce"] = asyncio.create_task(nodeOb.oTransmitter.announce())
await asyncio.sleep(1)
await n.s.app.start_server(port=int(sys.argv[2]), debug=True)
await nodeOb.server.app.start_server(port=int(sys.argv[2]), debug=True)
except KeyboardInterrupt:
sys.exit()
if __name__ == "__main__":
try:
t = threading.Thread(target=asyncio.run, args=(main(),))
t.start()
lplt = threading.Thread(target=asyncio.run, args=(logPassLoop(),))
lplt.start()
mainThread = threading.Thread(target=asyncio.run, args=(main(),))
mainThread.start()
lplThread = threading.Thread(target=asyncio.run, args=(logPassLoop(),))
lplThread.start()
tuiOb = TUI()
tuiOb.nodeOb = nodeOb
tuiOb.run()

View File

@ -4,13 +4,19 @@ from Components.daisy import Catch
from Components.daisy import Cache
import random
# TODO: Move intialization to run, this class is unnecessary
class Router:
"""
Router
"""
def __init__(self, cLog, nfpath="server.info"):
self.cLog = cLog
# TODO: Better network init
self.n = Network()
self.c = Catch(walk=True)
self.network = Network()
self.catch = Catch(walk=True)
self.cache = Cache(walk=True)
self.cLog(10, "Loading server info")
self.serverInfo = self.cache.get(nfpath)