ksbus cannot be used

[darcyg]

The example program example/client-go exits abnormally directly

client connected to ws://localhost:9313/ws/bus
CLIENT connected as go-client
ON OnDataWs: map[data:pong]
ERRO [ksbus/client.go:133] client handler for topic not found topic= time=02/12/2024 17:00
not received: 8ff57434-7fd4-45e0-877b-a46b82cb0bbc browser
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x6e6dec]

goroutine 1 [running]:
github.com/kamalshkeir/ksbus.Subscriber.Unsubscribe({0x0, {0x77e831, 0x9}, {0xc000244060, 0x24}, 0x0, 0xc000224000})
/works/ha-works/github/ksbus/subscriber.go:18 +0x4c
github.com/kamalshkeir/ksbus.(*Client).PublishToIDWaitRecv(0xc000132600, {0x77dece, 0x7}, 0xc0002083c0, 0x7aeeb8, 0x7aeec0)
/works/ha-works/github/ksbus/client.go:264 +0x2d4
main.main()
/works/ha-works/github/ksbus/examples/client-go/client.go:37 +0x2c7

---

After installing ksbus with pip3 install ksbus, the client-py demo cannot run.

python ksbus.py
Traceback (most recent call last):
File "ksbus.py", line 2, in
from ksbus import Bus
File "/works/ha-works/github/ksbus/examples/client-py/ksbus.py", line 2, in
from ksbus import Bus
ImportError: cannot import name 'Bus' from 'ksbus' (/works/ha-works/github/ksbus/examples/client-py/ksbus.py)

[kamalshkeir]

Hi @darcyg, let me check tonight, and i will be back to you

[darcyg]

Hi @darcyg, let me check tonight, and i will be back to you

Looking forward to testing your ksbus. It seems very good and exactly what I want. Thank you.

[kamalshkeir]

Should be fixed ;) @darcyg

[darcyg]

@kamalshkeir
The example has been successfully running.
I tried to modify the code to allow direct communication between Go and Python, such as changing the Go client ID to 'browser', but Python always reports an error.
Attempting to send data to the Go client using publish or publishWaitRecv also doesn't work.
Additionally, in the Python code, trying to remove block=true and send data also fails. Can you provide more example code?

[kamalshkeir]

@darcyg
i put the working example in python, it was because the server wasn't handling the waitrecv, but clients do, should be fixed, also should works with the new RPC Go Client.
i changed the serverID to master, and test publishToIDWaitRecv from python to go , and from go to python, working for both

make sure you update go get -u github.com/kamalshkeir/[email protected]
The block = true is to make the program block (if there is no server running beside it and blocking, like FastAPI for example), because it's async, also i know , there is a lot of errors when quitting the program, i will try to fix it later
Cheers

[darcyg]

@kamalshkeir
Could you provide the startup process and logs for a few programs?
I first load the server, then start the client-rpc, and finally launch the Python script.
However, the displayed information is quite different from what I expected. After looking at the source code, I feel that some normal process expectations are significantly different.
For example, the server subscribes to server1, but direct connections from Python do not work; I must immediately start the client-rpc.
Then, starting the client-py allows me to see the server1 information sent by the server.
Moreover, when starting client.py multiple times, only the first time can I see the server sending server1 information at 0.5s intervals.

This is significantly different from my expected pub/sub behavior. I believe that every time client-py connects, it should receive the server1 information within 50*0.5s.
Additionally, only the server has subscribed to server1, while both client-py and client-rpc are publishers. Why does client-rpc need to connect to the server?
Shouldn't client-py be able to work without connecting to the server?
thanks.

[kamalshkeir]

hi @darcyg
Should works like expected now.

Additionally, only the server has subscribed to server1, while both client-py and client-rpc are publishers. Why does client-rpc need to connect to the server?

you don't need any rpc client in this case, if you only need to communicated between python client and server bus Go

[darcyg]

@kamalshkeir
Sorry, I've had some things to take care of these past few days, so I'll conduct the tests later.
Are you familiar with the ubus in OpenWRT?

[kamalshkeir]

Hello again :), nope. first time i hear about it, what is it? Le jeu. 5 déc. 2024, 04:05, Darcy Gong ***@***.***> a écrit :

…

@kamalshkeir <https://github.com/kamalshkeir> Sorry, I've had some things to take care of these past few days, so I'll conduct the tests later. Are you familiar with the ubus in OpenWRT? — Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANATQT5BP7K7RMTDDGNLSLL2D67HFAVCNFSM6AAAAABS24LSTOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJYHE4DSMRYGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[darcyg]

@kamalshkeir 
Hi, I've tested your ksbus, and it seems to work now. I will further test it within the software, and I'm very grateful.
Ubus is a messaging bus used by OpenWRT, similar to dbus, but more lightweight.
It mainly uses a simplified protocol of JSON-RPC to allow OpenWRT's web framework (luci) and system command-line tools to handle inter-process communication and view different system configurations. I frequently used them between 2014 and 2016.
Ubus is primarily developed in C (with many third-party developments providing interfaces for C++, Python, and Go), and it has some basic system services bundled with OpenWRT's underlying features (used for routers). Moreover, the early ubus used a JSON-like data serialization scheme, which caused issues with handling bool/int/float values. So, although buildroot also provides packages for ubus, few people use it.
The ubus service under OpenWRT has a default rpcd process, which provides a multitude of default functionalities (out-of-the-box).
However, this rpcd is too tightly coupled with the design of OpenWRT. OpenWRT is an operating system designed for routers (based on NAT-based routing and forwarding). Therefore, it has some default network service dependencies (such as dhcpd, ntpd), which can be a significant hassle for general custom development based on buildroot (as those functionalities are not needed).
In the early days, OpenWRT required a minimal root file system (with the system ROM only being 2-4MB in size), so the early ubus RPC adopted a simplified JSON, rather than using standard JSON. This led to some issues with cross-language invocation and data conversion (for example, int only supports 32-bit, and anything beyond int32 had to be represented as a string).
I've always hoped to find a lightweight, customizable messaging bus protocol similar to ubus for web/cli/config/various script services.
One that could flexibly add features to the system using Go, Rust, and various scripting languages (Python/JavaScript/Lua/Shell).
For example, ksbus will have a standard ksbusd, mainly responsible for handling various RPC interfaces (providing WebSocket/Socket/UnixSocket/HTTP interfaces) without the need for user development.
There is also a command-line tool for ksbus that can list and view the methods provided by each mounted RPC module, supporting synchronous and asynchronous calls, as well as response calls (i.e., with return values).
RPC modules can be developed using Go/Rust or Python/JavaScript/Shell.
It can directly provide some basic out-of-the-box functionalities, such as operation interfaces for CPU/memory/disk/network/configuration management (which can be done with scripts).
The remaining user functionalities can be customized by themselves.

You can learn about the ubus and uci related features of OpenWRT.
I am still a novice in Go development, and in the future, I will mainly focus on compiled languages like Go and Rust, as well as development with various scripting languages.

[darcyg]

@kamalshkeir
Here's the translation into English:

I have already ported the ksbus service and RPC functionality into my code.
I am using the ABS scripting engine (https://github.com/abs-lang/abs), and the server and client have started normally and can respond as expected.
The current issue is that when I send a message every second, the server receives 2 messages, and the client displays 2 pieces of data.
Additionally, could you provide a stop method for ksbus.Server/ksbus.RPCClient (corresponding to run)?

my server code:
`
f on_serv_id(data) {
echo(data)
}

f on_serv_new_event(data) {
echo(data)
}

echo(ksbus_serv_init("master",":9313",":9314",on_serv_id))
echo(ksbus_serv_subscribe("new_event",on_serv_new_event))
echo(ksbus_serv_run())

while true {
echo("test")
sleep(1000)
}
`

my rpc client code:
`
f on_rpc_id(data) {
echo("rpc id: "+data)
}

f on_rpc_new_event(data) {
echo("rpc event: "+data)
}

echo(ksbus_rpc_init("rpc-client-imp","localhost:9314",false,on_rpc_id))
echo(ksbus_rpc_subscribe("new_event",on_rpc_new_event))
echo(ksbus_rpc_run())
c=0
while true {
#echo("test")
obj={
"xxx":c
}
ksbus_rpc_publish("new_event",obj.tojson())
sleep(1000)
c+=1
}
`

serv log:
darcy@hasee-z7m-ct7na:/works/ha-works/github/abs$ builds/abs test/test_rpc_server.abs
darcy@hasee-z7m-ct7na:/works/ha-works/github/abs$ builds/abs test/test_rpc_server.abs
true
true
running on http://localhost:9313
OnId: map[event_id:c7e527d4-ba13-45a3-8cb8-ad4bc5bcd506 from:rpc-client-imp id:master message:Hello from rpc client time:2024-12-09 16:07:38.428667781 +0800 CST m=+0.003416604 to_id:master]
map[event_id:c7e527d4-ba13-45a3-8cb8-ad4bc5bcd506 from:rpc-client-imp id:master message:Hello from rpc client time:2024-12-09 16:07:38.428667781 +0800 CST m=+0.003416604 to_id:master]
map[from:rpc-client-imp topic:new_event xxx:0]
map[from:rpc-client-imp topic:new_event xxx:0]
map[from:rpc-client-imp topic:new_event xxx:1]
map[from:rpc-client-imp topic:new_event xxx:1]
map[from:rpc-client-imp topic:new_event xxx:2]
map[from:rpc-client-imp topic:new_event xxx:2]
map[from:rpc-client-imp topic:new_event xxx:3]
map[from:rpc-client-imp topic:new_event xxx:3]
map[from:rpc-client-imp topic:new_event xxx:4]
map[from:rpc-client-imp topic:new_event xxx:4]

=========
rpc log:
darcy@hasee-z7m-ct7na:/works/ha-works/github/abs$ builds/abs test/test_rpc_client.abs
success map[from:master ok:done topic:1f1f7640-b63b-427e-a9bf-fe2e921b13e9]
true
true
true
rpc event: map[from:rpc-client-imp topic:new_event xxx:0]
rpc event: map[from:rpc-client-imp topic:new_event xxx:0]
rpc event: map[from:rpc-client-imp topic:new_event xxx:0]
rpc event: map[from:rpc-client-imp topic:new_event xxx:0]
rpc event: map[from:rpc-client-imp topic:new_event xxx:1]
rpc event: map[from:rpc-client-imp topic:new_event xxx:1]
rpc event: map[from:rpc-client-imp topic:new_event xxx:1]
rpc event: map[from:rpc-client-imp topic:new_event xxx:1]
rpc event: map[from:rpc-client-imp topic:new_event xxx:2]
rpc event: map[from:rpc-client-imp topic:new_event xxx:2]
rpc event: map[from:rpc-client-imp topic:new_event xxx:2]
rpc event: map[from:rpc-client-imp topic:new_event xxx:2]
^CINFO RPC Closed time=09/12/2024 16:08
RPC Client closeERRO [ksbus/rpc_client.go:334] connection is shut down time=09/12/2024 16:08
RPC Client closeRPC Client closeERRO error publishing topic=new_event err=connection is shut down time=09/12/2024 16:08
ERRO error publishing topic=new_event err=connection is shut down time=09/12/2024 16:08
ERRO error publishing topic=new_event err=connection is shut down time=09/12/2024 16:08
ERRO error publishing topic=new_event err=connection is shut down time=09/12/2024 16:08
^X^Z
[6]+ Stopped builds/abs test/test_rpc_client.abs

[kamalshkeir]

Hi @darcyg , should be fixed for duplicate messages RPC, bump ksbus version to v1.4.3.

and for Additionally, could you provide a stop method for ksbus.Server/ksbus.RPCClient (corresponding to run) , rpc client have already Close method, and server have Shutdown method, also you can use os.Exit to completely quit the program