Compare commits

..

2 Commits

Author SHA1 Message Date
mochi-co
d9c3de2c8c Add vendor to gitignore 2023-10-15 17:03:34 +01:00
mochi-co
787390e7ba Remove vendor folder 2023-10-14 18:59:59 +01:00
54 changed files with 341 additions and 2442 deletions

View File

@@ -11,12 +11,21 @@ RUN go mod download
COPY . ./
RUN go build -o /app/mochi ./cmd/docker
RUN go build -o /app/mochi ./cmd
FROM alpine
WORKDIR /
COPY --from=builder /app/mochi .
# tcp
EXPOSE 1883
# websockets
EXPOSE 1882
# dashboard
EXPOSE 8080
ENTRYPOINT [ "/mochi" ]
CMD ["/cmd/docker", "--config", "config.yaml"]

View File

@@ -10,7 +10,7 @@
</p>
[English](README.md) | [简体中文](README-CN.md) | [日本語](README-JP.md) | [招募翻译者!](https://github.com/orgs/mochi-mqtt/discussions/310)
[English](README.md) | [简体中文](README-CN.md) | [招募翻译者!](https://github.com/orgs/mochi-mqtt/discussions/310)
🎆 **mochi-co/mqtt 现在已经是新的 mochi-mqtt 组织的一部分。** 详细信息请[阅读公告.](https://github.com/orgs/mochi-mqtt/discussions/271)
@@ -183,7 +183,7 @@ server := mqtt.New(&mqtt.Options{
关于决定默认配置的值,在这里进行一些说明:
- 默认情况下server.Options.Capabilities.MaximumMessageExpiryInterval 的值被设置为 8640024小时以防止在使用默认配置时网络上暴露服务器而受到恶意DOS攻击如果不配置到期时间将允许无限数量的保留retained/待发送inflight消息累积。如果您在一个受信任的环境中运行或者您有更大的保留期容量您可以选择覆盖此设置设置为0 以取消到期限制)。
- 默认情况下server.Options.Capabilities.MaximumMessageExpiryInterval 的值被设置为 8640024小时以防止在使用默认配置时网络上暴露服务器而受到恶意DOS攻击如果不配置到期时间将允许无限数量的保留retained/待发送inflight消息累积。如果您在一个受信任的环境中运行或者您有更大的保留期容量您可以选择覆盖此设置设置为 0 或 math.MaxInt 以取消到期限制)。
## 事件钩子(Event Hooks)
@@ -200,7 +200,7 @@ server := mqtt.New(&mqtt.Options{
| 数据持久性 | [mochi-mqtt/server/hooks/storage/redis](hooks/storage/redis/redis.go) | 使用 [Redis](https://redis.io) 进行持久性存储。 |
| 调试跟踪 | [mochi-mqtt/server/hooks/debug](hooks/debug/debug.go) | 调试输出以查看数据包在服务端的链路追踪。 |
许多内部函数都已开放给开发者你可以参考上述示例创建自己的Hook钩子。如果你有更好的关于Hook钩子方面的建议或者疑问你可以[提交问题](https://github.com/mochi-mqtt/server/issues)给我们。
许多内部函数都已开放给开发者你可以参考上述示例创建自己的Hook钩子。如果你有更好的关于Hook钩子方面的建议或者疑问你可以[提交问题](https://github.com/mochi-mqtt/server/issues)给我们。 |
### 访问控制(Access Control)

View File

@@ -1,494 +0,0 @@
# Mochi-MQTT Server
<p align="center">
![build status](https://github.com/mochi-mqtt/server/actions/workflows/build.yml/badge.svg)
[![Coverage Status](https://coveralls.io/repos/github/mochi-mqtt/server/badge.svg?branch=master&v2)](https://coveralls.io/github/mochi-mqtt/server?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/mochi-mqtt/server)](https://goreportcard.com/report/github.com/mochi-mqtt/server/v2)
[![Go Reference](https://pkg.go.dev/badge/github.com/mochi-mqtt/server.svg)](https://pkg.go.dev/github.com/mochi-mqtt/server/v2)
[![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/mochi-mqtt/server/issues)
</p>
[English](README.md) | [简体中文](README-CN.md) | [日本語](README-JP.md) | [Translators Wanted!](https://github.com/orgs/mochi-mqtt/discussions/310)
🎆 **mochi-co/mqtt は新しい mochi-mqtt organisation の一部です.** [このページをお読みください](https://github.com/orgs/mochi-mqtt/discussions/271)
### Mochi-MQTTは MQTT v5 (と v3.1.1)に完全に準拠しているアプリケーションに組み込み可能なハイパフォーマンスなbroker/serverです.
Mochi MQTT は Goで書かれたMQTT v5に完全に[準拠](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html)しているMQTTブローカーで、IoTプロジェクトやテレメトリの開発プロジェクト向けに設計されています。 スタンドアロンのバイナリで使ったり、アプリケーションにライブラリとして組み込むことができ、プロジェクトのメンテナンス性と品質を確保できるように配慮しながら、 軽量で可能な限り速く動作するように設計されています。
#### MQTTとは?
MQTT は [MQ Telemetry Transport](https://en.wikipedia.org/wiki/MQTT)を意味します。 Pub/Sub型のシンプルで軽量なメッセージプロトコルで、低帯域、高遅延、不安定なネットワーク下での制約を考慮して設計されています([MQTTについて詳しくはこちら](https://mqtt.org/faq))。 Mochi MQTTはMQTTプロトコルv5.0.0に完全準拠した実装をしています。
#### Mochi-MQTTのもつ機能
- MQTTv5への完全な準拠とMQTT v3.1.1 および v3.0.0 との互換性:
- MQTT v5で拡張されたユーザープロパティ
- トピック・エイリアス
- 共有サブスクリプション
- サブスクリプションオプションとサブスクリプションID
- メッセージの有効期限
- クライアントセッション
- 送受信QoSフロー制御クォータ
- サーバサイド切断と認証パケット
- Will遅延間隔
- 上記に加えてQoS(0,1,2)、$SYSトピック、retain機能などすべてのMQTT v1の特徴を持ちます
- Developer-centric:
- 開発者が制御できるように、ほとんどのコアブローカーのコードをエクスポートにしてアクセスできるようにしました。
- フル機能で柔軟なフックベースのインターフェイスにすることで簡単に'プラグイン'を開発できるようにしました。
- 特別なインラインクライアントを利用することでパケットインジェクションを行うか、既存のクライアントとしてマスカレードすることができます。
- パフォーマンスと安定性:
- 古典的なツリーベースのトピックサブスクリプションモデル
- クライアント固有に書き込みバッファーをもたせることにより、読み込みの遅さや不規則なクライアントの挙動の問題を回避しています。
- MQTT v5 and MQTT v3のすべての[Paho互換性テスト](https://github.com/eclipse/paho.mqtt.testing/tree/master/interoperability)をpassしています。
- 慎重に検討された多くのユニットテストシナリオでテストされています。
- TCP, Websocket (SSL/TLSを含む), $SYSのダッシュボードリスナー
- フックを利用した保存機能としてRedis, Badger, Boltを使うことができます自作のHookも可能です
- フックを利用したルールベース認証機能とアクセス制御リストLedgerを使うことができます自作のHookも可能です
### 互換性に関する注意事項
MQTTv5とそれ以前との互換性から、サーバーはv5とv3両方のクライアントを受け入れることができますが、v5とv3のクライアントが接続された場合はv5でクライアント向けの特徴と機能はv3クライアントにダウングレードされますユーザープロパティなど
MQTT v3.0.0 と v3.1.1 のサポートはハイブリッド互換性があるとみなされます。それはv3と仕様に制限されていない場合、例えば、送信メッセージ、保持メッセージの有効期限とQoSフロー制御制限などについては、よりモダンで安全なv5の動作が使用されます
#### リリースされる時期について
クリティカルなイシュー出ない限り、新しいリリースがされるのは週末です。
## Roadmap
- 新しい特徴やイベントフックのリクエストは [open an issue](https://github.com/mochi-mqtt/server/issues) へ!
- クラスターのサポート
- メトリックスサポートの強化
- ファイルベースの設定(Dockerイメージのサポート)
## Quick Start
### GoでのBrokerの動かし方
Mochi MQTTはスタンドアロンのブローカーとして使うことができます。単純にこのレポジトリーをチェックアウトして、[cmd/main.go](cmd/main.go) を起動すると内部の [cmd](cmd) フォルダのエントリポイントにしてtcp (:1883), websocket (:1882), dashboard (:8080)のポートを外部にEXPOSEします。
```
cd cmd
go build -o mqtt && ./mqtt
```
### Dockerで利用する
Dockerレポジトリの [official Mochi MQTT image](https://hub.docker.com/r/mochimqtt/server) から Pullして起動することができます。
```sh
docker pull mochimqtt/server
or
docker run mochimqtt/server
```
これは実装途中です。[file-based configuration](https://github.com/orgs/mochi-mqtt/projects/2) は、この実装をよりよくサポートするために開発中です。
より実質的なdockerのサポートが議論されています。_Docker環境で使っている方は是非この議論に参加してください。_ [ここ](https://github.com/orgs/mochi-mqtt/discussions/281#discussion-5544545) や [ここ](https://github.com/orgs/mochi-mqtt/discussions/209)。
[cmd/main.go](cmd/main.go)の Websocket, TCP, Statsサーバを実行するために、シンプルなDockerfileが提供されます。
```sh
docker build -t mochi:latest .
docker run -p 1883:1883 -p 1882:1882 -p 8080:8080 mochi:latest
```
## Mochi MQTTを使って開発するには
### パッケージをインポート
Mochi MQTTをパッケージとしてインポートするにはほんの数行のコードで始めることができます。
``` go
import (
"log"
mqtt "github.com/mochi-mqtt/server/v2"
"github.com/mochi-mqtt/server/v2/hooks/auth"
"github.com/mochi-mqtt/server/v2/listeners"
)
func main() {
// Create signals channel to run server until interrupted
sigs := make(chan os.Signal, 1)
done := make(chan bool, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigs
done <- true
}()
// Create the new MQTT Server.
server := mqtt.New(nil)
// Allow all connections.
_ = server.AddHook(new(auth.AllowHook), nil)
// Create a TCP listener on a standard port.
tcp := listeners.NewTCP("t1", ":1883", nil)
err := server.AddListener(tcp)
if err != nil {
log.Fatal(err)
}
go func() {
err := server.Serve()
if err != nil {
log.Fatal(err)
}
}()
// Run server until interrupted
<-done
// Cleanup
}
```
ブローカーの動作例は [examples](examples)フォルダにあります。
#### Network Listeners
サーバは様々なプロトコルのコネクションのリスナーに対応しています。現在の対応リスナーは、
| Listener | Usage |
|------------------------------|----------------------------------------------------------------------------------------------|
| listeners.NewTCP | TCPリスナー |
| listeners.NewUnixSock | Unixソケットリスナー |
| listeners.NewNet | net.Listenerリスナー |
| listeners.NewWebsocket | Websocketリスナー |
| listeners.NewHTTPStats | HTTP $SYSダッシュボード |
| listeners.NewHTTPHealthCheck | ヘルスチェック応答を提供するためのHTTPヘルスチェックリスナー(クラウドインフラ) |
> 新しいリスナーを開発するためには `listeners.Listener` を使ってください。使ったら是非教えてください!
TLSを設定するには`*listeners.Config`を渡すことができます。
[examples](examples) フォルダと [cmd/main.go](cmd/main.go)に使用例があります。
## 設定できるオプションと機能
たくさんのオプションが利用可能です。サーバーの動作を変更したり、特定の機能へのアクセスを制限することができます。
```go
server := mqtt.New(&mqtt.Options{
Capabilities: mqtt.Capabilities{
MaximumSessionExpiryInterval: 3600,
Compatibilities: mqtt.Compatibilities{
ObscureNotAuthorized: true,
},
},
ClientNetWriteBufferSize: 4096,
ClientNetReadBufferSize: 4096,
SysTopicResendInterval: 10,
InlineClient: false,
})
```
mqtt.Options、mqtt.Capabilities、mqtt.Compatibilitiesの構造体はオプションの理解に役立ちます。
必要に応じて`ClientNetWriteBufferSize`と`ClientNetReadBufferSize`はクライアントの使用するメモリに合わせて設定できます。
### デフォルト設定に関する注意事項
いくつかのデフォルトの設定を決める際にいくつかの決定がなされましたのでここに記しておきます:
- デフォルトとして、敵対的なネットワーク上のDoSアタックにさらされるのを防ぐために `server.Options.Capabilities.MaximumMessageExpiryInterval`は86400 (24時間)に、とセットされています。有効期限を無限にすると、保持、送信メッセージが無限に蓄積されるからです。もし信頼できる環境であったり、より大きな保存期間が可能であれば、この設定はオーバーライドできます(`0` を設定すると有効期限はなくなります。)
## Event Hooks
ユニバーサルイベントフックシステムは、開発者にサーバとクライアントの様々なライフサイクルをフックすることができ、ブローカーの機能を追加/変更することができます。それらのユニバーサルフックは認証、永続ストレージ、デバッグツールなど、あらゆるものに使用されています。
フックは複数重ねることができ、サーバに複数のフックを設定することができます。それらは追加した順番に動作します。いくつかのフックは値を変えて、その値は動作コードに返される前にあとに続くフックに渡されます。
| Type | Import | Info |
|----------------|--------------------------------------------------------------------------|----------------------------------------------------------------------------|
| Access Control | [mochi-mqtt/server/hooks/auth . AllowHook](hooks/auth/allow_all.go) | すべてのトピックに対しての読み書きをすべてのクライアントに対して許可します。 |
| Access Control | [mochi-mqtt/server/hooks/auth . Auth](hooks/auth/auth.go) | ルールベースのアクセスコントロール台帳です。 |
| Persistence | [mochi-mqtt/server/hooks/storage/bolt](hooks/storage/bolt/bolt.go) | [BoltDB](https://dbdb.io/db/boltdb) を使った永続ストレージ (非推奨). |
| Persistence | [mochi-mqtt/server/hooks/storage/badger](hooks/storage/badger/badger.go) | [BadgerDB](https://github.com/dgraph-io/badger)を使った永続ストレージ |
| Persistence | [mochi-mqtt/server/hooks/storage/redis](hooks/storage/redis/redis.go) | [Redis](https://redis.io)を使った永続ストレージ |
| Debugging | [mochi-mqtt/server/hooks/debug](hooks/debug/debug.go) | パケットフローを可視化するデバッグ用のフック |
たくさんの内部関数が開発者に公開されています、なので、上記の例を使って自分でフックを作ることができます。もし作ったら是非[Open an issue](https://github.com/mochi-mqtt/server/issues)に投稿して教えてください!
### アクセスコントロール
#### Allow Hook
デフォルトで、Mochi MQTTはアクセスコントロールルールにDENY-ALLを使用しています。コネクションを許可するためには、アクセスコントロールフックを上書きする必要があります。一番単純なのは`auth.AllowAll`フックで、ALLOW-ALLルールがすべてのコネクション、サブスクリプション、パブリッシュに適用されます。使い方は下記のようにするだけです:
```go
server := mqtt.New(nil)
_ = server.AddHook(new(auth.AllowHook), nil)
```
> もしインターネットや信頼できないネットワークにさらされる場合は行わないでください。これは開発・テスト・デバッグ用途のみであるべきです。
#### Auth Ledger
Auth Ledgerは構造体で定義したアクセスルールの洗練された仕組みを提供します。Auth Ledgerルールつの形式から成ります、認証ルール(コネクション)とACLルール(パブリッシュ、サブスクライブ)です。
認証ルールは4つのクライテリアとアサーションフラグがあります:
| Criteria | Usage |
| -- | -- |
| Client | 接続クライアントのID |
| Username | 接続クライアントのユーザー名 |
| Password | 接続クライアントのパスワード |
| Remote | クライアントのリモートアドレスもしくはIP |
| Allow | true(このユーザーを許可する)もしくはfalse(このユーザを拒否する) |
アクセスコントロールルールは3つのクライテリアとフィルターマッチがあります:
| Criteria | Usage |
| -- | -- |
| Client | 接続クライアントのID |
| Username | 接続クライアントのユーザー名 |
| Remote | クライアントのリモートアドレスもしくはIP |
| Filters | 合致するフィルターの配列 |
ルールはインデックス順(0,1,2,3)に処理され、はじめに合致したルールが適用されます。 [hooks/auth/ledger.go](hooks/auth/ledger.go) の構造体を見てください。
```go
server := mqtt.New(nil)
err := server.AddHook(new(auth.Hook), &auth.Options{
Ledger: &auth.Ledger{
Auth: auth.AuthRules{ // Auth disallows all by default
{Username: "peach", Password: "password1", Allow: true},
{Username: "melon", Password: "password2", Allow: true},
{Remote: "127.0.0.1:*", Allow: true},
{Remote: "localhost:*", Allow: true},
},
ACL: auth.ACLRules{ // ACL allows all by default
{Remote: "127.0.0.1:*"}, // local superuser allow all
{
// user melon can read and write to their own topic
Username: "melon", Filters: auth.Filters{
"melon/#": auth.ReadWrite,
"updates/#": auth.WriteOnly, // can write to updates, but can't read updates from others
},
},
{
// Otherwise, no clients have publishing permissions
Filters: auth.Filters{
"#": auth.ReadOnly,
"updates/#": auth.Deny,
},
},
},
}
})
```
ledgeはデータフィールドを使用してJSONもしくはYAML形式で保存したものを使用することもできます。
```go
err := server.AddHook(new(auth.Hook), &auth.Options{
Data: data, // build ledger from byte slice: yaml or json
})
```
より詳しくは[examples/auth/encoded/main.go](examples/auth/encoded/main.go)を見てください。
### 永続ストレージ
#### Redis
ブローカーに永続性を提供する基本的な Redis ストレージフックが利用可能です。他のフックと同じ方法で、いくつかのオプションを使用してサーバーに追加できます。それはフック内部で github.com/go-redis/redis/v8 を使用し、Optionsの値で詳しい設定を行うことができます。
```go
err := server.AddHook(new(redis.Hook), &redis.Options{
Options: &rv8.Options{
Addr: "localhost:6379", // default redis address
Password: "", // your password
DB: 0, // your redis db
},
})
if err != nil {
log.Fatal(err)
}
```
Redisフックがどのように動くか、どのように使用するかについての詳しくは、[examples/persistence/redis/main.go](examples/persistence/redis/main.go) か [hooks/storage/redis](hooks/storage/redis) のソースコードを見てください。
#### Badger DB
もしファイルベースのストレージのほうが適しているのであれば、BadgerDBストレージも使用することができます。それもまた、他のフックと同様に追加、設定することができますオプションは若干少ないです
```go
err := server.AddHook(new(badger.Hook), &badger.Options{
Path: badgerPath,
})
if err != nil {
log.Fatal(err)
}
```
badgerフックがどのように動くか、どのように使用するかについての詳しくは、[examples/persistence/badger/main.go](examples/persistence/badger/main.go) か [hooks/storage/badger](hooks/storage/badger) のソースコードを見てください。
BoltDBフックはBadgerに代わって非推奨となりましたが、もし必要ならば [examples/persistence/bolt/main.go](examples/persistence/bolt/main.go)をチェックしてください。
## イベントフックを利用した開発
ブローカーとクライアントのライフサイクルに関わるたくさんのフックが利用できます。
そのすべてのフックと`mqtt.Hook`インターフェイスの関数シグネチャは[hooks.go](hooks.go)に記載されています。
> もっと柔軟なイベントフックはOnPacketRead、OnPacketEncodeとOnPacketSentです。それらは、すべての流入パケットと流出パケットをコントロール及び変更に使用されるフックです。
| Function | Usage |
|------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| OnStarted | サーバーが正常にスタートした際に呼ばれます。 |
| OnStopped | サーバーが正常に終了した際に呼ばれます。 |
| OnConnectAuthenticate | ユーザーがサーバと認証を試みた際に呼ばれます。このメソッドはサーバーへのアクセス許可もしくは拒否するためには必ず使用する必要がありますhooks/auth/allow_all or basicを見てください。これは、データベースにユーザーが存在するか照合してチェックするカスタムフックに利用できます。許可する場合はtrueを返す実装をします。|
| OnACLCheck | ユーザーがあるトピックフィルタにpublishかsubscribeした際に呼ばれます。上と同様です |
| OnSysInfoTick | $SYSトピック値がpublishされた場合に呼ばれます。 |
| OnConnect | 新しいクライアントが接続した際によばれます、エラーかパケットコードを返して切断する場合があります。 |
| OnSessionEstablish | 新しいクライアントが接続された後すぐ、セッションが確立されてCONNACKが送信される前に呼ばれます。 |
| OnSessionEstablished | 新しいクライアントがセッションを確立した際(OnConnectの後)に呼ばれます。 |
| OnDisconnect | クライアントが何らかの理由で切断された場合に呼ばれます。 |
| OnAuthPacket | 認証パケットを受け取ったときに呼ばれます。これは開発者にmqtt v5の認証パケットを取り扱う仕組みを作成すること意図しています。パケットを変更することができます。 |
| OnPacketRead | クライアントからパケットを受け取った際に呼ばれます。パケットを変更することができます。 |
| OnPacketEncode | エンコードされたパケットがクライアントに送信する直前に呼ばれます。パケットを変更することができます。 |
| OnPacketSent | クライアントにパケットが送信された際に呼ばれます。 |
| OnPacketProcessed | パケットが届いてブローカーが正しく処理できた場合に呼ばれます。 |
| OnSubscribe | クライアントがつ以上のフィルタをsubscribeした場合に呼ばれます。パケットの変更ができます。 |
| OnSubscribed | クライアントがつ以上のフィルタをsubscribeに成功した場合に呼ばれます。 |
| OnSelectSubscribers | サブスクライバーがトピックに収集されたとき、共有サブスクライバーが選択される前に呼ばれる。受信者は変更可能。 |
| OnUnsubscribe | 1つ以上のあんサブスクライブが呼ばれた場合。パケットの変更は可能。 |
| OnUnsubscribed | クライアントが正常に1つ以上のトピックフィルタをサブスクライブ解除した場合。 |
| OnPublish | クライアントがメッセージをパブリッシュした場合。パケットの変更は可能。 |
| OnPublished | クライアントがサブスクライバーにメッセージをパブリッシュし終わった場合。 |
| OnPublishDropped | あるクライアントが反応に時間がかかった場合等のようにクライアントに到達する前にメッセージが失われた場合に呼ばれる。 |
| OnRetainMessage | パブリッシュされたメッセージが保持された場合に呼ばれる。 |
| OnRetainPublished | 保持されたメッセージがクライアントに到達した場合に呼ばれる。 |
| OnQosPublish | QoSが1以上のパケットがサブスクライバーに発行された場合。 |
| OnQosComplete | そのメッセージQoSフローが完了した場合に呼ばれる。 |
| OnQosDropped | インフライトメッセージが完了前に期限切れになった場合に呼ばれる。 |
| OnPacketIDExhausted | クライアントがパケットに割り当てるIDが枯渇した場合に呼ばれる。 |
| OnWill | クライアントが切断し、WILLメッセージを発行しようとした場合に呼ばれる。パケットの変更が可能。 |
| OnWillSent | LWTメッセージが切断されたクライアントから発行された場合に呼ばれる |
| OnClientExpired | クライアントセッションが期限切れで削除するべき場合に呼ばれる。 |
| OnRetainedExpired | 保持メッセージが期限切れで削除すべき場合に呼ばれる。 |
| StoredClients | クライアントを返す。例えば永続ストレージから。 |
| StoredSubscriptions | クライアントのサブスクリプションを返す。例えば永続ストレージから。 |
| StoredInflightMessages | インフライトメッセージを返す。例えば永続ストレージから。 |
| StoredRetainedMessages | 保持されたメッセージを返す。例えば永続ストレージから。 |
| StoredSysInfo | システム情報の値を返す。例えば永続ストレージから。 |
もし永続ストレージフックを作成しようとしているのであれば、すでに存在する永続的なフックを見てインスピレーションとどのようなパターンがあるか見てみてください。もし認証フックを作成しようとしているのであれば、`OnACLCheck`と`OnConnectAuthenticate`が役立つでしょう。
### Inline Client (v2.4.0+)
トピックに対して埋め込まれたコードから直接サブスクライブとパブリッシュできます。そうするには`inline client`機能を使うことができます。インラインクライアント機能はサーバの一部として組み込まれているクライアントでサーバーのオプションとしてEnableにできます。
```go
server := mqtt.New(&mqtt.Options{
InlineClient: true,
})
```
Enableにすると、`server.Publish`, `server.Subscribe`, `server.Unsubscribe`のメソッドを利用できて、ブローカーから直接メッセージを送受信できます。
> 実際の使用例は[direct examples](examples/direct/main.go)を見てください。
#### Inline Publish
組み込まれたアプリケーションからメッセージをパブリッシュするには`server.Publish(topic string, payload []byte, retain bool, qos byte) error`メソッドを利用します。
```go
err := server.Publish("direct/publish", []byte("packet scheduled message"), false, 0)
```
> このケースでのQoSはサブスクライバーに設定できる上限でしか使用されません。これはMQTTv5の仕様に従っています。
#### Inline Subscribe
組み込まれたアプリケーション内部からトピックフィルタをサブスクライブするには、`server.Subscribe(filter string, subscriptionId int, handler InlineSubFn) error`メソッドがコールバックも含めて使用できます。
インラインサブスクリプションではQoS0のみが適用されます。もし複数のコールバックを同じフィルタに設定したい場合は、MQTTv5の`subscriptionId`のプロパティがその区別に使用できます。
```go
callbackFn := func(cl *mqtt.Client, sub packets.Subscription, pk packets.Packet) {
server.Log.Info("inline client received message from subscription", "client", cl.ID, "subscriptionId", sub.Identifier, "topic", pk.TopicName, "payload", string(pk.Payload))
}
server.Subscribe("direct/#", 1, callbackFn)
```
#### Inline Unsubscribe
インラインクライアントでサブスクリプション解除をしたい場合は、`server.Unsubscribe(filter string, subscriptionId int) error` メソッドで行うことができます。
```go
server.Unsubscribe("direct/#", 1)
```
### Packet Injection
もし、より制御したい場合や、特定のMQTTv5のプロパティやその他の値をセットしたい場合は、クライアントからのパブリッシュパケットを自ら作成することができます。この方法は単なるパブリッシュではなく、MQTTパケットをまるで特定のクライアントから受け取ったかのようにランタイムに直接インジェクションすることができます。
このパケットインジェクションは例えばPING ReqやサブスクリプションなどのどんなMQTTパケットでも使用できます。そしてクライアントの構造体とメソッドはエクスポートされているので、(もし、非常にカスタマイズ性の高い要求がある場合には)まるで接続されたクライアントに代わってパケットをインジェクションすることさえできます。
たいていの場合は上記のインラインクライアントを使用するのが良いでしょう、それはACLとトピックバリデーションをバイパスできる特権があるからです。これは$SYSトピックにさえパブリッシュできることも意味します。ビルトインのクライアントと同様に振る舞うインラインクライアントを作成できます。
```go
cl := server.NewClient(nil, "local", "inline", true)
server.InjectPacket(cl, packets.Packet{
FixedHeader: packets.FixedHeader{
Type: packets.Publish,
},
TopicName: "direct/publish",
Payload: []byte("scheduled message"),
})
```
> MQTTのパケットは正しく構成する必要があり、なので[the test packets catalogue](packets/tpackets.go)と[MQTTv5 Specification](https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html)を参照してください。
この機能の動作を確認するには[hooks example](examples/hooks/main.go) を見てください。
### Testing
#### ユニットテスト
それぞれの関数が期待通りの動作をするように考えられてMochi MQTTテストが作成されています。テストを走らせるには:
```
go run --cover ./...
```
#### Paho相互運用性テスト
`examples/paho/main.go`を使用してブローカーを起動し、_interoperability_フォルダの`python3 client_test5.py`のmqttv5とv3のテストを実行することで、[Paho Interoperability Test](https://github.com/eclipse/paho.mqtt.testing/tree/master/interoperability)を確認することができます。
> pahoスイートには現在は何個かの偽陰性に関わるissueがあるので、`paho/main.go`の例ではいくつかの互換性モードがオンになっていることに注意してください。
## ベンチマーク
Mochi MQTTのパフォーマンスはMosquitto、EMQX、その他などの有名なブローカーに匹敵します。
ベンチマークはApple Macbook Air M2上で[MQTT-Stresser](https://github.com/inovex/mqtt-stresser)、セッティングとして`cmd/main.go`のデフォルト設定を使用しています。高スループットと低スループットのバーストを考慮すると、中央値のスコアが最も信頼できます。この値は高いほど良いです。
> ベンチマークの値は1秒あたりのメッセージ数のスループットのそのものを表しているわけではありません。これは、mqtt-stresserによる固有の計算に依存するものではありますが、すべてのブローカーに渡って一貫性のある値として利用しています。
> ベンチマークは一般的なパフォーマンス予測ガイドラインとしてのみ提供されます。比較はそのまま使用したデフォルトの設定値で実行しています。
`mqtt-stresser -broker tcp://localhost:1883 -num-clients=2 -num-messages=10000`
| Broker | publish fastest | median | slowest | receive fastest | median | slowest |
| -- | -- | -- | -- | -- | -- | -- |
| Mochi v2.2.10 | 124,772 | 125,456 | 124,614 | 314,461 | 313,186 | 311,910 |
| [Mosquitto v2.0.15](https://github.com/eclipse/mosquitto) | 155,920 | 155,919 | 155,918 | 185,485 | 185,097 | 184,709 |
| [EMQX v5.0.11](https://github.com/emqx/emqx) | 156,945 | 156,257 | 155,568 | 17,918 | 17,783 | 17,649 |
| [Rumqtt v0.21.0](https://github.com/bytebeamio/rumqtt) | 112,208 | 108,480 | 104,753 | 135,784 | 126,446 | 117,108 |
`mqtt-stresser -broker tcp://localhost:1883 -num-clients=10 -num-messages=10000`
| Broker | publish fastest | median | slowest | receive fastest | median | slowest |
| -- | -- | -- | -- | -- | -- | -- |
| Mochi v2.2.10 | 41,825 | 31,663| 23,008 | 144,058 | 65,903 | 37,618 |
| Mosquitto v2.0.15 | 42,729 | 38,633 | 29,879 | 23,241 | 19,714 | 18,806 |
| EMQX v5.0.11 | 21,553 | 17,418 | 14,356 | 4,257 | 3,980 | 3,756 |
| Rumqtt v0.21.0 | 42,213 | 23,153 | 20,814 | 49,465 | 36,626 | 19,283 |
100万メッセージ試験 (100 万メッセージを一斉にサーバーに送信します):
`mqtt-stresser -broker tcp://localhost:1883 -num-clients=100 -num-messages=10000`
| Broker | publish fastest | median | slowest | receive fastest | median | slowest |
| -- | -- | -- | -- | -- | -- | -- |
| Mochi v2.2.10 | 13,532 | 4,425 | 2,344 | 52,120 | 7,274 | 2,701 |
| Mosquitto v2.0.15 | 3,826 | 3,395 | 3,032 | 1,200 | 1,150 | 1,118 |
| EMQX v5.0.11 | 4,086 | 2,432 | 2,274 | 434 | 333 | 311 |
| Rumqtt v0.21.0 | 78,972 | 5,047 | 3,804 | 4,286 | 3,249 | 2,027 |
> EMQXのここでの結果は何が起きているのかわかりませんが、おそらくDockerのそのままの設定が最適ではなかったのでしょう、なので、この結果はソフトウェアのひとつの側面にしか過ぎないと捉えてください。
## Contribution Guidelines
コントリビューションとフィードバックは両方とも歓迎しています![Open an issue](https://github.com/mochi-mqtt/server/issues)でバグを報告したり、質問したり、新機能のリクエストをしてください。もしプルリクエストするならば下記のガイドラインに従うようにしてください。
- 合理的で可能な限りテストカバレッジを維持してください
- なぜPRをしたのかとそのPRの内容について明確にしてください。
- 有意義な貢献をした場合はSPDX FileContributorタグをファイルにつけてください。
[SPDX Annotations](https://spdx.dev)はそのライセンス、著作権表記、コントリビューターについて明確するのために、それぞれのファイルに機械可読な形式で記されています。もし、新しいファイルをレポジトリに追加した場合は、下記のようなSPDXヘッダーを付与していることを確かめてください。
```go
// SPDX-License-Identifier: MIT
// SPDX-FileCopyrightText: 2023 mochi-mqtt
// SPDX-FileContributor: Your name or alias <optional@email.address>
package name
```
ファイルにそれぞれのコントリビューターの`SPDX-FileContributor`が追加されていることを確認してください、他のファイルを参考にしてください。あなたのこのプロジェクトへのコントリビュートは価値があり、高く評価されます!
## Stargazers over time 🥰
[![Stargazers over time](https://starchart.cc/mochi-mqtt/server.svg)](https://starchart.cc/mochi-mqtt/server)
Mochi MQTTをプロジェクトで使用していますか [是非私達に教えてください!](https://github.com/mochi-mqtt/server/issues)

View File

@@ -10,7 +10,7 @@
</p>
[English](README.md) | [简体中文](README-CN.md) | [日本語](README-JP.md) | [Translators Wanted!](https://github.com/orgs/mochi-mqtt/discussions/310)
[English](README.md) | [简体中文](README-CN.md) | [Translators Wanted!](https://github.com/orgs/mochi-mqtt/discussions/310)
🎆 **mochi-co/mqtt is now part of the new mochi-mqtt organisation.** [Read about this announcement here.](https://github.com/orgs/mochi-mqtt/discussions/271)
@@ -60,6 +60,7 @@ Unless it's a critical issue, new releases typically go out over the weekend.
- Please [open an issue](https://github.com/mochi-mqtt/server/issues) to request new features or event hooks!
- Cluster support.
- Enhanced Metrics support.
- File-based server configuration (supporting docker).
## Quick Start
### Running the Broker with Go
@@ -76,50 +77,18 @@ You can now pull and run the [official Mochi MQTT image](https://hub.docker.com/
```sh
docker pull mochimqtt/server
or
docker run -v $(pwd)/config.yaml:/config.yaml mochimqtt/server
docker run mochimqtt/server
```
For most use cases, you can use File Based Configuration to configure the server, by specifying a valid `yaml` or `json` config file.
This is a work in progress, and a [file-based configuration](https://github.com/orgs/mochi-mqtt/projects/2) is being developed to better support this implementation. _More substantial docker support is being discussed [here](https://github.com/orgs/mochi-mqtt/discussions/281#discussion-5544545) and [here](https://github.com/orgs/mochi-mqtt/discussions/209). Please join the discussion if you use Mochi-MQTT in this environment._
A simple Dockerfile is provided for running the [cmd/main.go](cmd/main.go) Websocket, TCP, and Stats server, using the `allow-all` auth hook.
A simple Dockerfile is provided for running the [cmd/main.go](cmd/main.go) Websocket, TCP, and Stats server:
```sh
docker build -t mochi:latest .
docker run -p 1883:1883 -p 1882:1882 -p 8080:8080 -v $(pwd)/config.yaml:/config.yaml mochi:latest
docker run -p 1883:1883 -p 1882:1882 -p 8080:8080 mochi:latest
```
### File Based Configuration
You can use File Based Configuration with either the Docker image (described above), or by running the build binary with the `--config=config.yaml` or `--config=config.json` parameter.
Configuration files provide a convenient mechanism for easily preparing a server with the most common configurations. You can enable and configure built-in hooks and listeners, and specify server options and compatibilities:
```yaml
listeners:
- type: "tcp"
id: "tcp12"
address: ":1883"
- type: "ws"
id: "ws1"
address: ":1882"
- type: "sysinfo"
id: "stats"
address: ":1880"
hooks:
auth:
allow_all: true
options:
inline_client: true
```
Please review the examples found in `examples/config` for all available configuration options.
There are a few conditions to note:
1. If you use file-based configuration, you can only have one of each hook type.
2. You can only use built in hooks with file-based configuration, as the type and configuration structure needs to be known by the server in order for it to be applied.
3. You can only use built in listeners, for the reasons above.
If you need to implement custom hooks or listeners, please do so using the traditional manner indicated in `cmd/main.go`.
## Developing with Mochi MQTT
### Importing as a package
Importing Mochi MQTT as a package requires just a few lines of code to get started.
@@ -215,7 +184,7 @@ Review the mqtt.Options, mqtt.Capabilities, and mqtt.Compatibilities structs for
Some choices were made when deciding the default configuration that need to be mentioned here:
- By default, the value of `server.Options.Capabilities.MaximumMessageExpiryInterval` is set to 86400 (24 hours), in order to prevent exposing the broker to DOS attacks on hostile networks when using the out-of-the-box configuration (as an infinite expiry would allow an infinite number of retained/inflight messages to accumulate). If you are operating in a trusted environment, or you have capacity for a larger retention period, you may wish to override this (set to `0` for no expiry).
- By default, the value of `server.Options.Capabilities.MaximumMessageExpiryInterval` is set to 86400 (24 hours), in order to prevent exposing the broker to DOS attacks on hostile networks when using the out-of-the-box configuration (as an infinite expiry would allow an infinite number of retained/inflight messages to accumulate). If you are operating in a trusted environment, or you have capacity for a larger retention period, uou may wish to override this (set to `0` or `math.MaxInt` for no expiry).
## Event Hooks
A universal event hooks system allows developers to hook into various parts of the server and client life cycle to add and modify functionality of the broker. These universal hooks are used to provide everything from authentication, persistent storage, to debugging tools.

View File

@@ -113,12 +113,11 @@ type Client struct {
// ClientConnection contains the connection transport and metadata for the client.
type ClientConnection struct {
Conn net.Conn // the net.Conn used to establish the connection
bconn *bufio.Reader // a buffered net.Conn for reading packets
outbuf *bytes.Buffer // a buffer for writing packets
Remote string // the remote address of the client
Listener string // listener id of the client
Inline bool // if true, the client is the built-in 'inline' embedded client
Conn net.Conn // the net.Conn used to establish the connection
bconn *bufio.ReadWriter // a buffered net.Conn for reading packets
Remote string // the remote address of the client
Listener string // listener id of the client
Inline bool // if true, the client is the built-in 'inline' embedded client
}
// ClientProperties contains the properties which define the client behaviour.
@@ -181,8 +180,11 @@ func newClient(c net.Conn, o *ops) *Client {
if c != nil {
cl.Net = ClientConnection{
Conn: c,
bconn: bufio.NewReaderSize(c, o.options.ClientNetReadBufferSize),
Conn: c,
bconn: bufio.NewReadWriter(
bufio.NewReaderSize(c, o.options.ClientNetReadBufferSize),
bufio.NewWriterSize(c, o.options.ClientNetWriteBufferSize),
),
Remote: c.RemoteAddr().String(),
}
}
@@ -215,10 +217,6 @@ func (cl *Client) ParseConnect(lid string, pk packets.Packet) {
cl.Properties.Clean = pk.Connect.Clean
cl.Properties.Props = pk.Properties.Copy(false)
if cl.Properties.Props.ReceiveMaximum > cl.ops.options.Capabilities.MaximumInflight { // 3.3.4 Non-normative
cl.Properties.Props.ReceiveMaximum = cl.ops.options.Capabilities.MaximumInflight
}
if pk.Connect.Keepalive <= minimumKeepalive {
cl.ops.log.Warn(
ErrMinimumKeepalive.Error(),
@@ -329,26 +327,10 @@ func (cl *Client) ResendInflightMessages(force bool) error {
}
// ClearInflights deletes all inflight messages for the client, e.g. for a disconnected user with a clean session.
func (cl *Client) ClearInflights() {
for _, tk := range cl.State.Inflight.GetAll(false) {
if ok := cl.State.Inflight.Delete(tk.PacketID); ok {
cl.ops.hooks.OnQosDropped(cl, tk)
atomic.AddInt64(&cl.ops.info.Inflight, -1)
}
}
}
// ClearExpiredInflights deletes any inflight messages which have expired.
func (cl *Client) ClearExpiredInflights(now, maximumExpiry int64) []uint16 {
func (cl *Client) ClearInflights(now, maximumExpiry int64) []uint16 {
deleted := []uint16{}
for _, tk := range cl.State.Inflight.GetAll(false) {
expired := tk.ProtocolVersion == 5 && tk.Expiry > 0 && tk.Expiry < now // [MQTT-3.3.2-5]
// If the maximum message expiry interval is set (greater than 0), and the message
// retention period exceeds the maximum expiry, the message will be forcibly removed.
enforced := maximumExpiry > 0 && now-tk.Created > maximumExpiry
if expired || enforced {
if (tk.Expiry > 0 && tk.Expiry < now) || tk.Created+maximumExpiry < now {
if ok := cl.State.Inflight.Delete(tk.PacketID); ok {
cl.ops.hooks.OnQosDropped(cl, tk)
atomic.AddInt64(&cl.ops.info.Inflight, -1)
@@ -586,35 +568,11 @@ func (cl *Client) WritePacket(pk packets.Packet) error {
return packets.ErrPacketTooLarge // [MQTT-3.1.2-24] [MQTT-3.1.2-25]
}
nb := net.Buffers{buf.Bytes()}
n, err := func() (int64, error) {
cl.Lock()
defer cl.Unlock()
if len(cl.State.outbound) == 0 {
if cl.Net.outbuf == nil {
return buf.WriteTo(cl.Net.Conn)
}
// first write to buffer, then flush buffer
n, _ := cl.Net.outbuf.Write(buf.Bytes()) // will always be successful
err = cl.flushOutbuf()
return int64(n), err
}
// there are more writes in the queue
if cl.Net.outbuf == nil {
if buf.Len() >= cl.ops.options.ClientNetWriteBufferSize {
return buf.WriteTo(cl.Net.Conn)
}
cl.Net.outbuf = new(bytes.Buffer)
}
n, _ := cl.Net.outbuf.Write(buf.Bytes()) // will always be successful
if cl.Net.outbuf.Len() < cl.ops.options.ClientNetWriteBufferSize {
return int64(n), nil
}
err = cl.flushOutbuf()
return int64(n), err
return nb.WriteTo(cl.Net.Conn)
}()
if err != nil {
return err
@@ -630,15 +588,3 @@ func (cl *Client) WritePacket(pk packets.Packet) error {
return err
}
func (cl *Client) flushOutbuf() (err error) {
if cl.Net.outbuf == nil {
return
}
_, err = cl.Net.outbuf.WriteTo(cl.Net.Conn)
if err == nil {
cl.Net.outbuf = nil
}
return
}

View File

@@ -37,7 +37,6 @@ func newTestClient() (cl *Client, r net.Conn, w net.Conn) {
options: &Options{
Capabilities: &Capabilities{
ReceiveMaximum: 10,
MaximumInflight: 5,
TopicAliasMaximum: 10000,
MaximumClientWritesPending: 3,
maximumPacketID: 10,
@@ -184,45 +183,6 @@ func TestClientParseConnect(t *testing.T) {
require.Equal(t, int32(pk.Properties.ReceiveMaximum), cl.State.Inflight.maximumSendQuota)
}
func TestClientParseConnectReceiveMaxExceedMaxInflight(t *testing.T) {
const MaxInflight uint16 = 1
cl, _, _ := newTestClient()
cl.ops.options.Capabilities.MaximumInflight = MaxInflight
pk := packets.Packet{
ProtocolVersion: 4,
Connect: packets.ConnectParams{
ProtocolName: []byte{'M', 'Q', 'T', 'T'},
Clean: true,
Keepalive: 60,
ClientIdentifier: "mochi",
WillFlag: true,
WillTopic: "lwt",
WillPayload: []byte("lol gg"),
WillQos: 1,
WillRetain: false,
},
Properties: packets.Properties{
ReceiveMaximum: uint16(5),
},
}
cl.ParseConnect("tcp1", pk)
require.Equal(t, pk.Connect.ClientIdentifier, cl.ID)
require.Equal(t, pk.Connect.Keepalive, cl.State.Keepalive)
require.Equal(t, pk.Connect.Clean, cl.Properties.Clean)
require.Equal(t, pk.Connect.ClientIdentifier, cl.ID)
require.Equal(t, pk.Connect.WillTopic, cl.Properties.Will.TopicName)
require.Equal(t, pk.Connect.WillPayload, cl.Properties.Will.Payload)
require.Equal(t, pk.Connect.WillQos, cl.Properties.Will.Qos)
require.Equal(t, pk.Connect.WillRetain, cl.Properties.Will.Retain)
require.Equal(t, uint32(1), cl.Properties.Will.Flag)
require.Equal(t, int32(cl.ops.options.Capabilities.ReceiveMaximum), cl.State.Inflight.receiveQuota)
require.Equal(t, int32(cl.ops.options.Capabilities.ReceiveMaximum), cl.State.Inflight.maximumReceiveQuota)
require.Equal(t, int32(MaxInflight), cl.State.Inflight.sendQuota)
require.Equal(t, int32(MaxInflight), cl.State.Inflight.maximumSendQuota)
}
func TestClientParseConnectOverrideWillDelay(t *testing.T) {
cl, _, _ := newTestClient()
@@ -342,56 +302,19 @@ func TestClientNextPacketIDOverflow(t *testing.T) {
func TestClientClearInflights(t *testing.T) {
cl, _, _ := newTestClient()
n := time.Now().Unix()
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 1, Expiry: n - 1})
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 2, Expiry: n - 2})
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 3, Created: n - 3}) // within bounds
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 5, Created: n - 5}) // over max server expiry limit
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 7, Created: n})
require.Equal(t, 5, cl.State.Inflight.Len())
cl.ClearInflights()
require.Equal(t, 0, cl.State.Inflight.Len())
}
func TestClientClearExpiredInflights(t *testing.T) {
cl, _, _ := newTestClient()
n := time.Now().Unix()
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 1, Expiry: n - 1})
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 2, Expiry: n - 2})
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 3, Created: n - 3}) // within bounds
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 5, Created: n - 5}) // over max server expiry limit
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 7, Created: n})
cl.State.Inflight.Set(packets.Packet{PacketID: 1, Expiry: n - 1})
cl.State.Inflight.Set(packets.Packet{PacketID: 2, Expiry: n - 2})
cl.State.Inflight.Set(packets.Packet{PacketID: 3, Created: n - 3}) // within bounds
cl.State.Inflight.Set(packets.Packet{PacketID: 5, Created: n - 5}) // over max server expiry limit
cl.State.Inflight.Set(packets.Packet{PacketID: 7, Created: n})
require.Equal(t, 5, cl.State.Inflight.Len())
deleted := cl.ClearExpiredInflights(n, 4)
deleted := cl.ClearInflights(n, 4)
require.Len(t, deleted, 3)
require.ElementsMatch(t, []uint16{1, 2, 5}, deleted)
require.Equal(t, 2, cl.State.Inflight.Len())
cl.State.Inflight.Set(packets.Packet{PacketID: 11, Expiry: n - 1})
cl.State.Inflight.Set(packets.Packet{PacketID: 12, Expiry: n - 2}) // expiry is ineffective for v3.
cl.State.Inflight.Set(packets.Packet{PacketID: 13, Created: n - 3}) // within bounds for v3
cl.State.Inflight.Set(packets.Packet{PacketID: 15, Created: n - 5}) // over max server expiry limit
require.Equal(t, 6, cl.State.Inflight.Len())
deleted = cl.ClearExpiredInflights(n, 4)
require.Len(t, deleted, 3)
require.ElementsMatch(t, []uint16{11, 12, 15}, deleted)
require.Equal(t, 3, cl.State.Inflight.Len())
cl.State.Inflight.Set(packets.Packet{PacketID: 17, Created: n - 1})
deleted = cl.ClearExpiredInflights(n, 0) // maximumExpiry = 0 do not process abandon messages
require.Len(t, deleted, 0)
require.Equal(t, 4, cl.State.Inflight.Len())
cl.State.Inflight.Set(packets.Packet{ProtocolVersion: 5, PacketID: 18, Expiry: n - 1})
deleted = cl.ClearExpiredInflights(n, 0) // maximumExpiry = 0 do not abandon messages
require.ElementsMatch(t, []uint16{18}, deleted) // expiry is still effective for v5.
require.Len(t, deleted, 1)
require.Equal(t, 4, cl.State.Inflight.Len())
}
func TestClientResendInflightMessages(t *testing.T) {
@@ -749,86 +672,6 @@ func TestClientWritePacket(t *testing.T) {
}
}
func TestClientWritePacketBuffer(t *testing.T) {
r, w := net.Pipe()
cl := newClient(w, &ops{
info: new(system.Info),
hooks: new(Hooks),
log: logger,
options: &Options{
Capabilities: &Capabilities{
ReceiveMaximum: 10,
TopicAliasMaximum: 10000,
MaximumClientWritesPending: 3,
maximumPacketID: 10,
},
},
})
cl.ID = "mochi"
cl.State.Inflight.maximumSendQuota = 5
cl.State.Inflight.sendQuota = 5
cl.State.Inflight.maximumReceiveQuota = 10
cl.State.Inflight.receiveQuota = 10
cl.Properties.Props.TopicAliasMaximum = 0
cl.Properties.Props.RequestResponseInfo = 0x1
cl.ops.options.ClientNetWriteBufferSize = 10
defer cl.Stop(errClientStop)
small := packets.TPacketData[packets.Publish].Get(packets.TPublishNoPayload).Packet
large := packets.TPacketData[packets.Publish].Get(packets.TPublishBasic).Packet
cl.State.outbound <- small
tt := []struct {
pks []*packets.Packet
size int
}{
{
pks: []*packets.Packet{small, small},
size: 18,
},
{
pks: []*packets.Packet{large},
size: 20,
},
{
pks: []*packets.Packet{small},
size: 0,
},
}
go func() {
for i, tx := range tt {
for _, pk := range tx.pks {
cl.Properties.ProtocolVersion = pk.ProtocolVersion
err := cl.WritePacket(*pk)
require.NoError(t, err, "index: %d", i)
if i == len(tt)-1 {
cl.Net.Conn.Close()
}
time.Sleep(100 * time.Millisecond)
}
}
}()
var n int
var err error
for i, tx := range tt {
buf := make([]byte, 100)
if i == len(tt)-1 {
buf, err = io.ReadAll(r)
n = len(buf)
} else {
n, err = io.ReadAtLeast(r, buf, 1)
}
require.NoError(t, err, "index: %d", i)
require.Equal(t, tx.size, n, "index: %d", i)
}
}
func TestWriteClientOversizePacket(t *testing.T) {
cl, _, _ := newTestClient()
cl.Properties.Props.MaximumPacketSize = 2

View File

@@ -1,66 +0,0 @@
// SPDX-License-Identifier: MIT
// SPDX-FileCopyrightText: 2023 mochi-mqtt
// SPDX-FileContributor: dgduncan, mochi-co
package main
import (
"flag"
"fmt"
"github.com/mochi-mqtt/server/v2/config"
"log"
"log/slog"
"os"
"os/signal"
"syscall"
mqtt "github.com/mochi-mqtt/server/v2"
)
func main() {
slog.SetDefault(slog.New(slog.NewTextHandler(os.Stdout, nil))) // set basic logger to ensure logs before configuration are in a consistent format
configFile := flag.String("config", "config.yaml", "path to mochi config yaml or json file")
flag.Parse()
entries, err := os.ReadDir("./")
if err != nil {
log.Fatal(err)
}
for _, e := range entries {
fmt.Println(e.Name())
}
sigs := make(chan os.Signal, 1)
done := make(chan bool, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigs
done <- true
}()
configBytes, err := os.ReadFile(*configFile)
if err != nil {
log.Fatal(err)
}
options, err := config.FromBytes(configBytes)
if err != nil {
log.Fatal(err)
}
server := mqtt.New(options)
go func() {
err := server.Serve()
if err != nil {
log.Fatal(err)
}
}()
<-done
server.Log.Warn("caught signal, stopping...")
_ = server.Close()
server.Log.Info("mochi mqtt shutdown complete")
}

View File

@@ -33,31 +33,19 @@ func main() {
server := mqtt.New(nil)
_ = server.AddHook(new(auth.AllowHook), nil)
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: *tcpAddr,
})
tcp := listeners.NewTCP("t1", *tcpAddr, nil)
err := server.AddListener(tcp)
if err != nil {
log.Fatal(err)
}
ws := listeners.NewWebsocket(listeners.Config{
ID: "ws1",
Address: *wsAddr,
})
ws := listeners.NewWebsocket("ws1", *wsAddr, nil)
err = server.AddListener(ws)
if err != nil {
log.Fatal(err)
}
stats := listeners.NewHTTPStats(
listeners.Config{
ID: "info",
Address: *infoAddr,
},
server.Info,
)
stats := listeners.NewHTTPStats("stats", *infoAddr, nil, server.Info)
err = server.AddListener(stats)
if err != nil {
log.Fatal(err)
@@ -73,5 +61,6 @@ func main() {
<-done
server.Log.Warn("caught signal, stopping...")
_ = server.Close()
server.Log.Info("mochi mqtt shutdown complete")
server.Log.Info("main.go finished")
}

View File

@@ -1,15 +0,0 @@
listeners:
- type: "tcp"
id: "tcp12"
address: ":1883"
- type: "ws"
id: "ws1"
address: ":1882"
- type: "sysinfo"
id: "stats"
address: ":1880"
hooks:
auth:
allow_all: true
options:
inline_client: true

View File

@@ -1,144 +0,0 @@
// SPDX-License-Identifier: MIT
// SPDX-FileCopyrightText: 2023 mochi-mqtt, mochi-co
// SPDX-FileContributor: mochi-co
package config
import (
"encoding/json"
"github.com/mochi-mqtt/server/v2/hooks/auth"
"github.com/mochi-mqtt/server/v2/hooks/debug"
"github.com/mochi-mqtt/server/v2/hooks/storage/badger"
"github.com/mochi-mqtt/server/v2/hooks/storage/bolt"
"github.com/mochi-mqtt/server/v2/hooks/storage/redis"
"github.com/mochi-mqtt/server/v2/listeners"
"gopkg.in/yaml.v3"
mqtt "github.com/mochi-mqtt/server/v2"
)
// config defines the structure of configuration data to be parsed from a config source.
type config struct {
Options mqtt.Options
Listeners []listeners.Config `yaml:"listeners" json:"listeners"`
HookConfigs HookConfigs `yaml:"hooks" json:"hooks"`
}
// HookConfigs contains configurations to enable individual hooks.
type HookConfigs struct {
Auth *HookAuthConfig `yaml:"auth" json:"auth"`
Storage *HookStorageConfig `yaml:"storage" json:"storage"`
Debug *debug.Options `yaml:"debug" json:"debug"`
}
// HookAuthConfig contains configurations for the auth hook.
type HookAuthConfig struct {
Ledger auth.Ledger `yaml:"ledger" json:"ledger"`
AllowAll bool `yaml:"allow_all" json:"allow_all"`
}
// HookStorageConfig contains configurations for the different storage hooks.
type HookStorageConfig struct {
Badger *badger.Options `yaml:"badger" json:"badger"`
Bolt *bolt.Options `yaml:"bolt" json:"bolt"`
Redis *redis.Options `yaml:"redis" json:"redis"`
}
// ToHooks converts Hook file configurations into Hooks to be added to the server.
func (hc HookConfigs) ToHooks() []mqtt.HookLoadConfig {
var hlc []mqtt.HookLoadConfig
if hc.Auth != nil {
hlc = append(hlc, hc.toHooksAuth()...)
}
if hc.Storage != nil {
hlc = append(hlc, hc.toHooksStorage()...)
}
if hc.Debug != nil {
hlc = append(hlc, mqtt.HookLoadConfig{
Hook: new(debug.Hook),
Config: hc.Debug,
})
}
return hlc
}
// toHooksAuth converts auth hook configurations into auth hooks.
func (hc HookConfigs) toHooksAuth() []mqtt.HookLoadConfig {
var hlc []mqtt.HookLoadConfig
if hc.Auth.AllowAll {
hlc = append(hlc, mqtt.HookLoadConfig{
Hook: new(auth.AllowHook),
})
} else {
hlc = append(hlc, mqtt.HookLoadConfig{
Hook: new(auth.Hook),
Config: &auth.Options{
Ledger: &auth.Ledger{ // avoid copying sync.Locker
Users: hc.Auth.Ledger.Users,
Auth: hc.Auth.Ledger.Auth,
ACL: hc.Auth.Ledger.ACL,
},
},
})
}
return hlc
}
// toHooksAuth converts storage hook configurations into storage hooks.
func (hc HookConfigs) toHooksStorage() []mqtt.HookLoadConfig {
var hlc []mqtt.HookLoadConfig
if hc.Storage.Badger != nil {
hlc = append(hlc, mqtt.HookLoadConfig{
Hook: new(badger.Hook),
Config: hc.Storage.Badger,
})
}
if hc.Storage.Bolt != nil {
hlc = append(hlc, mqtt.HookLoadConfig{
Hook: new(bolt.Hook),
Config: hc.Storage.Bolt,
})
}
if hc.Storage.Redis != nil {
hlc = append(hlc, mqtt.HookLoadConfig{
Hook: new(redis.Hook),
Config: hc.Storage.Redis,
})
}
return hlc
}
// FromBytes unmarshals a byte slice of JSON or YAML config data into a valid server options value.
// Any hooks configurations are converted into Hooks using the toHooks methods in this package.
func FromBytes(b []byte) (*mqtt.Options, error) {
c := new(config)
o := mqtt.Options{}
if len(b) == 0 {
return nil, nil
}
if b[0] == '{' {
err := json.Unmarshal(b, c)
if err != nil {
return nil, err
}
} else {
err := yaml.Unmarshal(b, c)
if err != nil {
return nil, err
}
}
o = c.Options
o.Hooks = c.HookConfigs.ToHooks()
o.Listeners = c.Listeners
return &o, nil
}

View File

@@ -1,212 +0,0 @@
// SPDX-License-Identifier: MIT
// SPDX-FileCopyrightText: 2023 mochi-mqtt, mochi-co
// SPDX-FileContributor: mochi-co
package config
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/mochi-mqtt/server/v2/hooks/auth"
"github.com/mochi-mqtt/server/v2/hooks/storage/badger"
"github.com/mochi-mqtt/server/v2/hooks/storage/bolt"
"github.com/mochi-mqtt/server/v2/hooks/storage/redis"
"github.com/mochi-mqtt/server/v2/listeners"
mqtt "github.com/mochi-mqtt/server/v2"
)
var (
yamlBytes = []byte(`
listeners:
- type: "tcp"
id: "file-tcp1"
address: ":1883"
hooks:
auth:
allow_all: true
options:
client_net_write_buffer_size: 2048
capabilities:
minimum_protocol_version: 3
compatibilities:
restore_sys_info_on_restart: true
`)
jsonBytes = []byte(`{
"listeners": [
{
"type": "tcp",
"id": "file-tcp1",
"address": ":1883"
}
],
"hooks": {
"auth": {
"allow_all": true
}
},
"options": {
"client_net_write_buffer_size": 2048,
"capabilities": {
"minimum_protocol_version": 3,
"compatibilities": {
"restore_sys_info_on_restart": true
}
}
}
}
`)
parsedOptions = mqtt.Options{
Listeners: []listeners.Config{
{
Type: listeners.TypeTCP,
ID: "file-tcp1",
Address: ":1883",
},
},
Hooks: []mqtt.HookLoadConfig{
{
Hook: new(auth.AllowHook),
},
},
ClientNetWriteBufferSize: 2048,
Capabilities: &mqtt.Capabilities{
MinimumProtocolVersion: 3,
Compatibilities: mqtt.Compatibilities{
RestoreSysInfoOnRestart: true,
},
},
}
)
func TestFromBytesEmptyL(t *testing.T) {
_, err := FromBytes([]byte{})
require.NoError(t, err)
}
func TestFromBytesYAML(t *testing.T) {
o, err := FromBytes(yamlBytes)
require.NoError(t, err)
require.Equal(t, parsedOptions, *o)
}
func TestFromBytesYAMLError(t *testing.T) {
_, err := FromBytes(append(yamlBytes, 'a'))
require.Error(t, err)
}
func TestFromBytesJSON(t *testing.T) {
o, err := FromBytes(jsonBytes)
require.NoError(t, err)
require.Equal(t, parsedOptions, *o)
}
func TestFromBytesJSONError(t *testing.T) {
_, err := FromBytes(append(jsonBytes, 'a'))
require.Error(t, err)
}
func TestToHooksAuthAllowAll(t *testing.T) {
hc := HookConfigs{
Auth: &HookAuthConfig{
AllowAll: true,
},
}
th := hc.toHooksAuth()
expect := []mqtt.HookLoadConfig{
{Hook: new(auth.AllowHook)},
}
require.Equal(t, expect, th)
}
func TestToHooksAuthAllowLedger(t *testing.T) {
hc := HookConfigs{
Auth: &HookAuthConfig{
Ledger: auth.Ledger{
Auth: auth.AuthRules{
{Username: "peach", Password: "password1", Allow: true},
},
},
},
}
th := hc.toHooksAuth()
expect := []mqtt.HookLoadConfig{
{
Hook: new(auth.Hook),
Config: &auth.Options{
Ledger: &auth.Ledger{ // avoid copying sync.Locker
Auth: auth.AuthRules{
{Username: "peach", Password: "password1", Allow: true},
},
},
},
},
}
require.Equal(t, expect, th)
}
func TestToHooksStorageBadger(t *testing.T) {
hc := HookConfigs{
Storage: &HookStorageConfig{
Badger: &badger.Options{
Path: "badger",
},
},
}
th := hc.toHooksStorage()
expect := []mqtt.HookLoadConfig{
{
Hook: new(badger.Hook),
Config: hc.Storage.Badger,
},
}
require.Equal(t, expect, th)
}
func TestToHooksStorageBolt(t *testing.T) {
hc := HookConfigs{
Storage: &HookStorageConfig{
Bolt: &bolt.Options{
Path: "bolt",
},
},
}
th := hc.toHooksStorage()
expect := []mqtt.HookLoadConfig{
{
Hook: new(bolt.Hook),
Config: hc.Storage.Bolt,
},
}
require.Equal(t, expect, th)
}
func TestToHooksStorageRedis(t *testing.T) {
hc := HookConfigs{
Storage: &HookStorageConfig{
Redis: &redis.Options{
Username: "test",
},
},
}
th := hc.toHooksStorage()
expect := []mqtt.HookLoadConfig{
{
Hook: new(redis.Hook),
Config: hc.Storage.Redis,
},
}
require.Equal(t, expect, th)
}

View File

@@ -63,10 +63,7 @@ func main() {
log.Fatal(err)
}
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err = server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -45,10 +45,7 @@ func main() {
log.Fatal(err)
}
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err = server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -32,10 +32,7 @@ func main() {
server.Options.Capabilities.MaximumClientWritesPending = 16 * 1024
_ = server.AddHook(new(auth.AllowHook), nil)
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: *tcpAddr,
})
tcp := listeners.NewTCP("t1", *tcpAddr, nil)
err := server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -1,92 +0,0 @@
{
"listeners": [
{
"type": "tcp",
"id": "file-tcp1",
"address": ":1883"
},
{
"type": "ws",
"id": "file-websocket",
"address": ":1882"
},
{
"type": "healthcheck",
"id": "file-healthcheck",
"address": ":1880"
}
],
"hooks": {
"debug": {
"enable": true
},
"storage": {
"badger": {
"path": "badger.db",
"gc_interval": 3,
"gc_discard_ratio": 0.5
},
"bolt": {
"path": "bolt.db"
},
"redis": {
"h_prefix": "mc",
"username": "mochi",
"password": "melon",
"address": "localhost:6379",
"database": 1
}
},
"auth": {
"allow_all": false,
"ledger": {
"auth": [
{
"username": "peach",
"password": "password1",
"allow": true
}
],
"acl": [
{
"remote": "127.0.0.1:*"
},
{
"username": "melon",
"filters": null,
"melon/#": 3,
"updates/#": 2
}
]
}
}
},
"options": {
"client_net_write_buffer_size": 2048,
"client_net_read_buffer_size": 2048,
"sys_topic_resend_interval": 10,
"inline_client": true,
"capabilities": {
"maximum_message_expiry_interval": 100,
"maximum_client_writes_pending": 8192,
"maximum_session_expiry_interval": 86400,
"maximum_packet_size": 0,
"receive_maximum": 1024,
"maximum_inflight": 8192,
"topic_alias_maximum": 65535,
"shared_sub_available": 1,
"minimum_protocol_version": 3,
"maximum_qos": 2,
"retain_available": 1,
"wildcard_sub_available": 1,
"sub_id_available": 1,
"compatibilities": {
"obscure_not_authorized": true,
"passive_client_disconnect": false,
"always_return_response_info": false,
"restore_sys_info_on_restart": false,
"no_inherited_properties_on_ack": false
}
}
}
}

View File

@@ -1,64 +0,0 @@
listeners:
- type: "tcp"
id: "file-tcp1"
address: ":1883"
- type: "ws"
id: "file-websocket"
address: ":1882"
- type: "healthcheck"
id: "file-healthcheck"
address: ":1880"
hooks:
debug:
enable: true
storage:
badger:
path: badger.db
gc_interval: 3
gc_discard_ratio: 0.5
bolt:
path: bolt.db
redis:
h_prefix: "mc"
username: "mochi"
password: "melon"
address: "localhost:6379"
database: 1
auth:
allow_all: false
ledger:
auth:
- username: peach
password: password1
allow: true
acl:
- remote: 127.0.0.1:*
- username: melon
filters:
melon/#: 3
updates/#: 2
options:
client_net_write_buffer_size: 2048
client_net_read_buffer_size: 2048
sys_topic_resend_interval: 10
inline_client: true
capabilities:
maximum_message_expiry_interval: 100
maximum_client_writes_pending: 8192
maximum_session_expiry_interval: 86400
maximum_packet_size: 0
receive_maximum: 1024
maximum_inflight: 8192
topic_alias_maximum: 65535
shared_sub_available: 1
minimum_protocol_version: 3
maximum_qos: 2
retain_available: 1
wildcard_sub_available: 1
sub_id_available: 1
compatibilities:
obscure_not_authorized: true
passive_client_disconnect: false
always_return_response_info: false
restore_sys_info_on_restart: false
no_inherited_properties_on_ack: false

View File

@@ -1,49 +0,0 @@
// SPDX-License-Identifier: MIT
// SPDX-FileCopyrightText: 2023 mochi-mqtt, mochi-co
// SPDX-FileContributor: mochi-co
package main
import (
"github.com/mochi-mqtt/server/v2/config"
"log"
"os"
"os/signal"
"syscall"
mqtt "github.com/mochi-mqtt/server/v2"
)
func main() {
sigs := make(chan os.Signal, 1)
done := make(chan bool, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigs
done <- true
}()
configBytes, err := os.ReadFile("config.json")
if err != nil {
log.Fatal(err)
}
options, err := config.FromBytes(configBytes)
if err != nil {
log.Fatal(err)
}
server := mqtt.New(options)
go func() {
err := server.Serve()
if err != nil {
log.Fatal(err)
}
}()
<-done
server.Log.Warn("caught signal, stopping...")
_ = server.Close()
server.Log.Info("main.go finished")
}

View File

@@ -46,10 +46,7 @@ func main() {
log.Fatal(err)
}
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err = server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -28,25 +28,15 @@ func main() {
done <- true
}()
server := mqtt.New(&mqtt.Options{
InlineClient: true, // you must enable inline client to use direct publishing and subscribing.
})
server := mqtt.New(nil)
_ = server.AddHook(new(auth.AllowHook), nil)
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err := server.AddListener(tcp)
if err != nil {
log.Fatal(err)
}
// Add custom hook (ExampleHook) to the server
err = server.AddHook(new(ExampleHook), &ExampleHookOptions{
Server: server,
})
err = server.AddHook(new(ExampleHook), map[string]any{})
if err != nil {
log.Fatal(err)
}
@@ -97,14 +87,8 @@ func main() {
server.Log.Info("main.go finished")
}
// Options contains configuration settings for the hook.
type ExampleHookOptions struct {
Server *mqtt.Server
}
type ExampleHook struct {
mqtt.HookBase
config *ExampleHookOptions
}
func (h *ExampleHook) ID() string {
@@ -124,34 +108,11 @@ func (h *ExampleHook) Provides(b byte) bool {
func (h *ExampleHook) Init(config any) error {
h.Log.Info("initialised")
if _, ok := config.(*ExampleHookOptions); !ok && config != nil {
return mqtt.ErrInvalidConfigType
}
h.config = config.(*ExampleHookOptions)
if h.config.Server == nil {
return mqtt.ErrInvalidConfigType
}
return nil
}
// subscribeCallback handles messages for subscribed topics
func (h *ExampleHook) subscribeCallback(cl *mqtt.Client, sub packets.Subscription, pk packets.Packet) {
h.Log.Info("hook subscribed message", "client", cl.ID, "topic", pk.TopicName)
}
func (h *ExampleHook) OnConnect(cl *mqtt.Client, pk packets.Packet) error {
h.Log.Info("client connected", "client", cl.ID)
// Example demonstrating how to subscribe to a topic within the hook.
h.config.Server.Subscribe("hook/direct/publish", 1, h.subscribeCallback)
// Example demonstrating how to publish a message within the hook
err := h.config.Server.Publish("hook/direct/publish", []byte("packet hook message"), false, 0)
if err != nil {
h.Log.Error("hook.publish", "error", err)
}
return nil
}

View File

@@ -31,10 +31,7 @@ func main() {
server.Options.Capabilities.Compatibilities.NoInheritedPropertiesOnAck = true
_ = server.AddHook(new(pahoAuthHook), nil)
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err := server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -5,16 +5,15 @@
package main
import (
badgerdb "github.com/dgraph-io/badger"
mqtt "github.com/mochi-mqtt/server/v2"
"github.com/mochi-mqtt/server/v2/hooks/auth"
"github.com/mochi-mqtt/server/v2/hooks/storage/badger"
"github.com/mochi-mqtt/server/v2/listeners"
"github.com/timshannon/badgerhold"
"log"
"os"
"os/signal"
"syscall"
mqtt "github.com/mochi-mqtt/server/v2"
"github.com/mochi-mqtt/server/v2/hooks/auth"
"github.com/mochi-mqtt/server/v2/hooks/storage/badger"
"github.com/mochi-mqtt/server/v2/listeners"
)
func main() {
@@ -32,39 +31,14 @@ func main() {
server := mqtt.New(nil)
_ = server.AddHook(new(auth.AllowHook), nil)
// AddHook adds a BadgerDB hook to the server with the specified options.
// GcInterval specifies the interval at which BadgerDB garbage collection process runs.
// Refer to https://dgraph.io/docs/badger/get-started/#garbage-collection for more information.
err := server.AddHook(new(badger.Hook), &badger.Options{
Path: badgerPath,
// Set the interval for garbage collection. Adjust according to your actual scenario.
GcInterval: 5 * 60,
// GcDiscardRatio specifies the ratio of log discard compared to the maximum possible log discard.
// Setting it to a higher value would result in fewer space reclaims, while setting it to a lower value
// would result in more space reclaims at the cost of increased activity on the LSM tree.
// discardRatio must be in the range (0.0, 1.0), both endpoints excluded, otherwise, it will be set to the default value of 0.5.
// Adjust according to your actual scenario.
GcDiscardRatio: 0.5,
Options: &badgerhold.Options{
// BadgerDB options. Adjust according to your actual scenario.
Options: badgerdb.Options{
NumCompactors: 2, // Number of compactors. Compactions can be expensive.
MaxTableSize: 64 << 20, // Maximum size of each table (64 MB).
ValueLogFileSize: 100 * (1 << 20), // Set the default size of the log file to 100 MB.
},
},
})
if err != nil {
log.Fatal(err)
}
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err = server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -40,10 +40,7 @@ func main() {
log.Fatal(err)
}
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err = server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -48,10 +48,7 @@ func main() {
log.Fatal(err)
}
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err = server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -38,10 +38,7 @@ func main() {
log.Fatal(err)
}
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
})
tcp := listeners.NewTCP("t1", ":1883", nil)
err = server.AddListener(tcp)
if err != nil {
log.Fatal(err)

View File

@@ -79,9 +79,7 @@ func main() {
server := mqtt.New(nil)
_ = server.AddHook(new(auth.AllowHook), nil)
tcp := listeners.NewTCP(listeners.Config{
ID: "t1",
Address: ":1883",
tcp := listeners.NewTCP("t1", ":1883", &listeners.Config{
TLSConfig: tlsConfig,
})
err = server.AddListener(tcp)
@@ -89,9 +87,7 @@ func main() {
log.Fatal(err)
}
ws := listeners.NewWebsocket(listeners.Config{
ID: "ws1",
Address: ":1882",
ws := listeners.NewWebsocket("ws1", ":1882", &listeners.Config{
TLSConfig: tlsConfig,
})
err = server.AddListener(ws)
@@ -99,13 +95,9 @@ func main() {
log.Fatal(err)
}
stats := listeners.NewHTTPStats(
listeners.Config{
ID: "stats",
Address: ":8080",
TLSConfig: tlsConfig,
}, server.Info,
)
stats := listeners.NewHTTPStats("stats", ":8080", &listeners.Config{
TLSConfig: tlsConfig,
}, server.Info)
err = server.AddListener(stats)
if err != nil {
log.Fatal(err)

View File

@@ -27,10 +27,7 @@ func main() {
server := mqtt.New(nil)
_ = server.AddHook(new(auth.AllowHook), nil)
ws := listeners.NewWebsocket(listeners.Config{
ID: "ws1",
Address: ":1882",
})
ws := listeners.NewWebsocket("ws1", ":1882", nil)
err := server.AddListener(ws)
if err != nil {
log.Fatal(err)

2
go.mod
View File

@@ -33,5 +33,5 @@ require (
github.com/yuin/gopher-lua v0.0.0-20210529063254-f4c35e4016d9 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/sys v0.13.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
google.golang.org/protobuf v1.28.1 // indirect
)

4
go.sum
View File

@@ -124,8 +124,8 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T
google.golang.org/appengine v1.6.5 h1:tycE03LOZYQNhDpS27tcQdAzLCVMaj7QT2SXxebnpCM=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w=
google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=

View File

@@ -62,12 +62,6 @@ var (
ErrInvalidConfigType = errors.New("invalid config type provided")
)
// HookLoadConfig contains the hook and configuration as loaded from a configuration (usually file).
type HookLoadConfig struct {
Hook Hook
Config any
}
// Hook provides an interface of handlers for different events which occur
// during the lifecycle of the broker.
type Hook interface {
@@ -76,7 +70,6 @@ type Hook interface {
Init(config any) error
Stop() error
SetOpts(l *slog.Logger, o *HookOptions)
OnStarted()
OnStopped()
OnConnectAuthenticate(cl *Client, pk packets.Packet) bool

View File

@@ -16,10 +16,9 @@ import (
// Options contains configuration settings for the debug output.
type Options struct {
Enable bool `yaml:"enable" json:"enable"` // non-zero field for enabling hook using file-based config
ShowPacketData bool `yaml:"show_packet_data" json:"show_packet_data"` // include decoded packet data (default false)
ShowPings bool `yaml:"show_pings" json:"show_pings"` // show ping requests and responses (default false)
ShowPasswords bool `yaml:"show_passwords" json:"show_passwords"` // show connecting user passwords (default false)
ShowPacketData bool // include decoded packet data (default false)
ShowPings bool // show ping requests and responses (default false)
ShowPasswords bool // show connecting user passwords (default false)
}
// Hook is a debugging hook which logs additional low-level information from the server.

View File

@@ -9,7 +9,6 @@ import (
"errors"
"fmt"
"strings"
"time"
mqtt "github.com/mochi-mqtt/server/v2"
"github.com/mochi-mqtt/server/v2/hooks/storage"
@@ -21,9 +20,7 @@ import (
const (
// defaultDbFile is the default file path for the badger db file.
defaultDbFile = ".badger"
defaultGcInterval = 5 * 60 // gc interval in seconds
defaultGcDiscardRatio = 0.5
defaultDbFile = ".badger"
)
// clientKey returns a primary key for a client.
@@ -54,21 +51,14 @@ func sysInfoKey() string {
// Options contains configuration settings for the BadgerDB instance.
type Options struct {
Options *badgerhold.Options
Path string `yaml:"path" json:"path"`
// GcDiscardRatio specifies the ratio of log discard compared to the maximum possible log discard.
// Setting it to a higher value would result in fewer space reclaims, while setting it to a lower value
// would result in more space reclaims at the cost of increased activity on the LSM tree.
// discardRatio must be in the range (0.0, 1.0), both endpoints excluded, otherwise, it will be set to the default value of 0.5.
GcDiscardRatio float64 `yaml:"gc_discard_ratio" json:"gc_discard_ratio"`
GcInterval int64 `yaml:"gc_interval" json:"gc_interval"`
Path string
}
// Hook is a persistent storage hook based using BadgerDB file store as a backend.
type Hook struct {
mqtt.HookBase
config *Options // options for configuring the BadgerDB instance.
gcTicker *time.Ticker // Ticker for BadgerDB garbage collection.
db *badgerhold.Store // the BadgerDB instance.
config *Options // options for configuring the BadgerDB instance.
db *badgerhold.Store // the BadgerDB instance.
}
// ID returns the id of the hook.
@@ -99,21 +89,6 @@ func (h *Hook) Provides(b byte) bool {
}, []byte{b})
}
// GcLoop periodically runs the garbage collection process to reclaim space in the value log files.
// It uses a ticker to trigger the garbage collection at regular intervals specified by the configuration.
// Refer to: https://dgraph.io/docs/badger/get-started/#garbage-collection
func (h *Hook) GcLoop() {
for range h.gcTicker.C {
again:
// Run the garbage collection process with a threshold.
// If the process returns nil (success), repeat the process.
err := h.db.Badger().RunValueLogGC(h.config.GcDiscardRatio)
if err == nil {
goto again // Retry garbage collection if successful.
}
}
}
// Init initializes and connects to the badger instance.
func (h *Hook) Init(config any) error {
if _, ok := config.(*Options); !ok && config != nil {
@@ -129,14 +104,6 @@ func (h *Hook) Init(config any) error {
h.config.Path = defaultDbFile
}
if h.config.GcInterval == 0 {
h.config.GcInterval = defaultGcInterval
}
if h.config.GcDiscardRatio <= 0.0 || h.config.GcDiscardRatio >= 1.0 {
h.config.GcDiscardRatio = defaultGcDiscardRatio
}
options := badgerhold.DefaultOptions
options.Dir = h.config.Path
options.ValueDir = h.config.Path
@@ -148,17 +115,11 @@ func (h *Hook) Init(config any) error {
return err
}
h.gcTicker = time.NewTicker(time.Duration(h.config.GcInterval) * time.Second)
go h.GcLoop()
return nil
}
// Stop closes the badger instance.
func (h *Hook) Stop() error {
if h.gcTicker != nil {
h.gcTicker.Stop()
}
return h.db.Close()
}
@@ -221,7 +182,7 @@ func (h *Hook) OnDisconnect(cl *mqtt.Client, _ error, expire bool) {
return
}
if errors.Is(cl.StopCause(), packets.ErrSessionTakenOver) {
if cl.StopCause() == packets.ErrSessionTakenOver {
return
}

View File

@@ -11,7 +11,6 @@ import (
"testing"
"time"
badgerdb "github.com/dgraph-io/badger"
mqtt "github.com/mochi-mqtt/server/v2"
"github.com/mochi-mqtt/server/v2/hooks/storage"
"github.com/mochi-mqtt/server/v2/packets"
@@ -703,21 +702,3 @@ func TestDebugf(t *testing.T) {
h.SetOpts(logger, nil)
h.Debugf("test", 1, 2, 3)
}
func TestGcLoop(t *testing.T) {
h := new(Hook)
h.SetOpts(logger, nil)
h.Init(&Options{
GcInterval: 2, // Set the interval for garbage collection.
Options: &badgerhold.Options{
// BadgerDB options. Modify as needed.
Options: badgerdb.Options{
ValueLogFileSize: 1 << 20, // Set the default size of the log file to 1 MB.
},
},
})
defer teardown(t, h.config.Path, h)
h.OnSessionEstablished(client, packets.Packet{})
h.OnDisconnect(client, nil, true)
time.Sleep(3 * time.Second)
}

View File

@@ -56,7 +56,7 @@ func sysInfoKey() string {
// Options contains configuration settings for the bolt instance.
type Options struct {
Options *bbolt.Options
Path string `yaml:"path" json:"path"`
Path string
}
// Hook is a persistent storage hook based using boltdb file store as a backend.

View File

@@ -51,12 +51,8 @@ func sysInfoKey() string {
// Options contains configuration settings for the bolt instance.
type Options struct {
Address string `yaml:"address" json:"address"`
Username string `yaml:"username" json:"username"`
Password string `yaml:"password" json:"password"`
Database int `yaml:"database" json:"database"`
HPrefix string `yaml:"h_prefix" json:"h_prefix"`
Options *redis.Options
HPrefix string
Options *redis.Options
}
// Hook is a persistent storage hook based using Redis as a backend.
@@ -109,31 +105,23 @@ func (h *Hook) Init(config any) error {
h.ctx = context.Background()
if config == nil {
config = new(Options)
}
h.config = config.(*Options)
if h.config.Options == nil {
h.config.Options = &redis.Options{
Addr: defaultAddr,
config = &Options{
Options: &redis.Options{
Addr: defaultAddr,
},
}
h.config.Options.Addr = h.config.Address
h.config.Options.DB = h.config.Database
h.config.Options.Username = h.config.Username
h.config.Options.Password = h.config.Password
}
h.config = config.(*Options)
if h.config.HPrefix == "" {
h.config.HPrefix = defaultHPrefix
}
h.Log.Info(
"connecting to redis service",
"prefix", h.config.HPrefix,
h.Log.Info("connecting to redis service",
"address", h.config.Options.Addr,
"username", h.config.Options.Username,
"password-len", len(h.config.Options.Password),
"db", h.config.Options.DB,
)
"db", h.config.Options.DB)
h.db = redis.NewClient(h.config.Options)
_, err := h.db.Ping(context.Background()).Result()

View File

@@ -135,29 +135,6 @@ func TestInitUseDefaults(t *testing.T) {
require.Equal(t, defaultAddr, h.config.Options.Addr)
}
func TestInitUsePassConfig(t *testing.T) {
s := miniredis.RunT(t)
s.StartAddr(defaultAddr)
defer s.Close()
h := newHook(t, "")
h.SetOpts(logger, nil)
err := h.Init(&Options{
Address: defaultAddr,
Username: "username",
Password: "password",
Database: 2,
})
require.Error(t, err)
h.db.FlushAll(h.ctx)
require.Equal(t, defaultAddr, h.config.Options.Addr)
require.Equal(t, "username", h.config.Options.Username)
require.Equal(t, "password", h.config.Options.Password)
require.Equal(t, 2, h.config.Options.DB)
}
func TestInitBadConfig(t *testing.T) {
h := new(Hook)
h.SetOpts(logger, nil)

View File

@@ -13,23 +13,24 @@ import (
"time"
)
const TypeHealthCheck = "healthcheck"
// HTTPHealthCheck is a listener for providing an HTTP healthcheck endpoint.
type HTTPHealthCheck struct {
sync.RWMutex
id string // the internal id of the listener
address string // the network address to bind to
config Config // configuration values for the listener
config *Config // configuration values for the listener
listen *http.Server // the http server
end uint32 // ensure the close methods are only called once
}
// NewHTTPHealthCheck initializes and returns a new HTTP listener, listening on an address.
func NewHTTPHealthCheck(config Config) *HTTPHealthCheck {
// NewHTTPHealthCheck initialises and returns a new HTTP listener, listening on an address.
func NewHTTPHealthCheck(id, address string, config *Config) *HTTPHealthCheck {
if config == nil {
config = new(Config)
}
return &HTTPHealthCheck{
id: config.ID,
address: config.Address,
id: id,
address: address,
config: config,
}
}

View File

@@ -14,44 +14,47 @@ import (
)
func TestNewHTTPHealthCheck(t *testing.T) {
l := NewHTTPHealthCheck(basicConfig)
require.Equal(t, basicConfig.ID, l.id)
require.Equal(t, basicConfig.Address, l.address)
l := NewHTTPHealthCheck("healthcheck", testAddr, nil)
require.Equal(t, "healthcheck", l.id)
require.Equal(t, testAddr, l.address)
}
func TestHTTPHealthCheckID(t *testing.T) {
l := NewHTTPHealthCheck(basicConfig)
require.Equal(t, basicConfig.ID, l.ID())
l := NewHTTPHealthCheck("healthcheck", testAddr, nil)
require.Equal(t, "healthcheck", l.ID())
}
func TestHTTPHealthCheckAddress(t *testing.T) {
l := NewHTTPHealthCheck(basicConfig)
require.Equal(t, basicConfig.Address, l.Address())
l := NewHTTPHealthCheck("healthcheck", testAddr, nil)
require.Equal(t, testAddr, l.Address())
}
func TestHTTPHealthCheckProtocol(t *testing.T) {
l := NewHTTPHealthCheck(basicConfig)
l := NewHTTPHealthCheck("healthcheck", testAddr, nil)
require.Equal(t, "http", l.Protocol())
}
func TestHTTPHealthCheckTLSProtocol(t *testing.T) {
l := NewHTTPHealthCheck(tlsConfig)
l := NewHTTPHealthCheck("healthcheck", testAddr, &Config{
TLSConfig: tlsConfigBasic,
})
_ = l.Init(logger)
require.Equal(t, "https", l.Protocol())
}
func TestHTTPHealthCheckInit(t *testing.T) {
l := NewHTTPHealthCheck(basicConfig)
l := NewHTTPHealthCheck("healthcheck", testAddr, nil)
err := l.Init(logger)
require.NoError(t, err)
require.NotNil(t, l.listen)
require.Equal(t, basicConfig.Address, l.listen.Addr)
require.Equal(t, testAddr, l.listen.Addr)
}
func TestHTTPHealthCheckServeAndClose(t *testing.T) {
// setup http stats listener
l := NewHTTPHealthCheck(basicConfig)
l := NewHTTPHealthCheck("healthcheck", testAddr, nil)
err := l.Init(logger)
require.NoError(t, err)
@@ -87,7 +90,7 @@ func TestHTTPHealthCheckServeAndClose(t *testing.T) {
func TestHTTPHealthCheckServeAndCloseMethodNotAllowed(t *testing.T) {
// setup http stats listener
l := NewHTTPHealthCheck(basicConfig)
l := NewHTTPHealthCheck("healthcheck", testAddr, nil)
err := l.Init(logger)
require.NoError(t, err)
@@ -122,7 +125,10 @@ func TestHTTPHealthCheckServeAndCloseMethodNotAllowed(t *testing.T) {
}
func TestHTTPHealthCheckServeTLSAndClose(t *testing.T) {
l := NewHTTPHealthCheck(tlsConfig)
l := NewHTTPHealthCheck("healthcheck", testAddr, &Config{
TLSConfig: tlsConfigBasic,
})
err := l.Init(logger)
require.NoError(t, err)

View File

@@ -17,26 +17,27 @@ import (
"github.com/mochi-mqtt/server/v2/system"
)
const TypeSysInfo = "sysinfo"
// HTTPStats is a listener for presenting the server $SYS stats on a JSON http endpoint.
type HTTPStats struct {
sync.RWMutex
id string // the internal id of the listener
address string // the network address to bind to
config Config // configuration values for the listener
config *Config // configuration values for the listener
listen *http.Server // the http server
sysInfo *system.Info // pointers to the server data
log *slog.Logger // server logger
end uint32 // ensure the close methods are only called once
}
// NewHTTPStats initializes and returns a new HTTP listener, listening on an address.
func NewHTTPStats(config Config, sysInfo *system.Info) *HTTPStats {
// NewHTTPStats initialises and returns a new HTTP listener, listening on an address.
func NewHTTPStats(id, address string, config *Config, sysInfo *system.Info) *HTTPStats {
if config == nil {
config = new(Config)
}
return &HTTPStats{
id: id,
address: address,
sysInfo: sysInfo,
id: config.ID,
address: config.Address,
config: config,
}
}

View File

@@ -17,35 +17,38 @@ import (
)
func TestNewHTTPStats(t *testing.T) {
l := NewHTTPStats(basicConfig, nil)
l := NewHTTPStats("t1", testAddr, nil, nil)
require.Equal(t, "t1", l.id)
require.Equal(t, testAddr, l.address)
}
func TestHTTPStatsID(t *testing.T) {
l := NewHTTPStats(basicConfig, nil)
l := NewHTTPStats("t1", testAddr, nil, nil)
require.Equal(t, "t1", l.ID())
}
func TestHTTPStatsAddress(t *testing.T) {
l := NewHTTPStats(basicConfig, nil)
l := NewHTTPStats("t1", testAddr, nil, nil)
require.Equal(t, testAddr, l.Address())
}
func TestHTTPStatsProtocol(t *testing.T) {
l := NewHTTPStats(basicConfig, nil)
l := NewHTTPStats("t1", testAddr, nil, nil)
require.Equal(t, "http", l.Protocol())
}
func TestHTTPStatsTLSProtocol(t *testing.T) {
l := NewHTTPStats(tlsConfig, nil)
l := NewHTTPStats("t1", testAddr, &Config{
TLSConfig: tlsConfigBasic,
}, nil)
_ = l.Init(logger)
require.Equal(t, "https", l.Protocol())
}
func TestHTTPStatsInit(t *testing.T) {
sysInfo := new(system.Info)
l := NewHTTPStats(basicConfig, sysInfo)
l := NewHTTPStats("t1", testAddr, nil, sysInfo)
err := l.Init(logger)
require.NoError(t, err)
@@ -61,7 +64,7 @@ func TestHTTPStatsServeAndClose(t *testing.T) {
}
// setup http stats listener
l := NewHTTPStats(basicConfig, sysInfo)
l := NewHTTPStats("t1", testAddr, nil, sysInfo)
err := l.Init(logger)
require.NoError(t, err)
@@ -106,7 +109,9 @@ func TestHTTPStatsServeTLSAndClose(t *testing.T) {
Version: "test",
}
l := NewHTTPStats(tlsConfig, sysInfo)
l := NewHTTPStats("t1", testAddr, &Config{
TLSConfig: tlsConfigBasic,
}, sysInfo)
err := l.Init(logger)
require.NoError(t, err)
@@ -127,9 +132,7 @@ func TestHTTPStatsFailedToServe(t *testing.T) {
}
// setup http stats listener
config := basicConfig
config.Address = "wrong_addr"
l := NewHTTPStats(config, sysInfo)
l := NewHTTPStats("t1", "wrong_addr", nil, sysInfo)
err := l.Init(logger)
require.NoError(t, err)

View File

@@ -14,10 +14,8 @@ import (
// Config contains configuration values for a listener.
type Config struct {
Type string
ID string
Address string
// TLSConfig is a tls.Config configuration to be used with the listener. See examples folder for basic and mutual-tls use.
// TLSConfig is a tls.Config configuration to be used with the listener.
// See examples folder for basic and mutual-tls use.
TLSConfig *tls.Config
}

View File

@@ -19,9 +19,6 @@ import (
const testAddr = ":22222"
var (
basicConfig = Config{ID: "t1", Address: testAddr}
tlsConfig = Config{ID: "t1", Address: testAddr, TLSConfig: tlsConfigBasic}
logger = slog.New(slog.NewTextHandler(os.Stdout, nil))
testCertificate = []byte(`-----BEGIN CERTIFICATE-----
@@ -68,7 +65,6 @@ func init() {
MinVersion: tls.VersionTLS12,
Certificates: []tls.Certificate{cert},
}
tlsConfig.TLSConfig = tlsConfigBasic
}
func TestNew(t *testing.T) {

View File

@@ -12,8 +12,6 @@ import (
"log/slog"
)
const TypeMock = "mock"
// MockEstablisher is a function signature which can be used in testing.
func MockEstablisher(id string, c net.Conn) error {
return nil

View File

@@ -13,24 +13,26 @@ import (
"log/slog"
)
const TypeTCP = "tcp"
// TCP is a listener for establishing client connections on basic TCP protocol.
type TCP struct { // [MQTT-4.2.0-1]
sync.RWMutex
id string // the internal id of the listener
address string // the network address to bind to
listen net.Listener // a net.Listener which will listen for new clients
config Config // configuration values for the listener
config *Config // configuration values for the listener
log *slog.Logger // server logger
end uint32 // ensure the close methods are only called once
}
// NewTCP initializes and returns a new TCP listener, listening on an address.
func NewTCP(config Config) *TCP {
// NewTCP initialises and returns a new TCP listener, listening on an address.
func NewTCP(id, address string, config *Config) *TCP {
if config == nil {
config = new(Config)
}
return &TCP{
id: config.ID,
address: config.Address,
id: id,
address: address,
config: config,
}
}
@@ -42,9 +44,6 @@ func (l *TCP) ID() string {
// Address returns the address of the listener.
func (l *TCP) Address() string {
if l.listen != nil {
return l.listen.Addr().String()
}
return l.address
}

View File

@@ -14,40 +14,45 @@ import (
)
func TestNewTCP(t *testing.T) {
l := NewTCP(basicConfig)
l := NewTCP("t1", testAddr, nil)
require.Equal(t, "t1", l.id)
require.Equal(t, testAddr, l.address)
}
func TestTCPID(t *testing.T) {
l := NewTCP(basicConfig)
l := NewTCP("t1", testAddr, nil)
require.Equal(t, "t1", l.ID())
}
func TestTCPAddress(t *testing.T) {
l := NewTCP(basicConfig)
l := NewTCP("t1", testAddr, nil)
require.Equal(t, testAddr, l.Address())
}
func TestTCPProtocol(t *testing.T) {
l := NewTCP(basicConfig)
l := NewTCP("t1", testAddr, nil)
require.Equal(t, "tcp", l.Protocol())
}
func TestTCPProtocolTLS(t *testing.T) {
l := NewTCP(tlsConfig)
l := NewTCP("t1", testAddr, &Config{
TLSConfig: tlsConfigBasic,
})
_ = l.Init(logger)
defer l.listen.Close()
require.Equal(t, "tcp", l.Protocol())
}
func TestTCPInit(t *testing.T) {
l := NewTCP(basicConfig)
l := NewTCP("t1", testAddr, nil)
err := l.Init(logger)
l.Close(MockCloser)
require.NoError(t, err)
l2 := NewTCP(tlsConfig)
l2 := NewTCP("t2", testAddr, &Config{
TLSConfig: tlsConfigBasic,
})
err = l2.Init(logger)
l2.Close(MockCloser)
require.NoError(t, err)
@@ -55,7 +60,7 @@ func TestTCPInit(t *testing.T) {
}
func TestTCPServeAndClose(t *testing.T) {
l := NewTCP(basicConfig)
l := NewTCP("t1", testAddr, nil)
err := l.Init(logger)
require.NoError(t, err)
@@ -80,7 +85,9 @@ func TestTCPServeAndClose(t *testing.T) {
}
func TestTCPServeTLSAndClose(t *testing.T) {
l := NewTCP(tlsConfig)
l := NewTCP("t1", testAddr, &Config{
TLSConfig: tlsConfigBasic,
})
err := l.Init(logger)
require.NoError(t, err)
@@ -102,7 +109,7 @@ func TestTCPServeTLSAndClose(t *testing.T) {
}
func TestTCPEstablishThenEnd(t *testing.T) {
l := NewTCP(basicConfig)
l := NewTCP("t1", testAddr, nil)
err := l.Init(logger)
require.NoError(t, err)

View File

@@ -13,25 +13,21 @@ import (
"log/slog"
)
const TypeUnix = "unix"
// UnixSock is a listener for establishing client connections on basic UnixSock protocol.
type UnixSock struct {
sync.RWMutex
id string // the internal id of the listener.
address string // the network address to bind to.
config Config // configuration values for the listener
listen net.Listener // a net.Listener which will listen for new clients.
log *slog.Logger // server logger
end uint32 // ensure the close methods are only called once.
}
// NewUnixSock initializes and returns a new UnixSock listener, listening on an address.
func NewUnixSock(config Config) *UnixSock {
// NewUnixSock initialises and returns a new UnixSock listener, listening on an address.
func NewUnixSock(id, address string) *UnixSock {
return &UnixSock{
id: config.ID,
address: config.Address,
config: config,
id: id,
address: address,
}
}

View File

@@ -15,47 +15,41 @@ import (
const testUnixAddr = "mochi.sock"
var (
unixConfig = Config{ID: "t1", Address: testUnixAddr}
)
func TestNewUnixSock(t *testing.T) {
l := NewUnixSock(unixConfig)
l := NewUnixSock("t1", testUnixAddr)
require.Equal(t, "t1", l.id)
require.Equal(t, testUnixAddr, l.address)
}
func TestUnixSockID(t *testing.T) {
l := NewUnixSock(unixConfig)
l := NewUnixSock("t1", testUnixAddr)
require.Equal(t, "t1", l.ID())
}
func TestUnixSockAddress(t *testing.T) {
l := NewUnixSock(unixConfig)
l := NewUnixSock("t1", testUnixAddr)
require.Equal(t, testUnixAddr, l.Address())
}
func TestUnixSockProtocol(t *testing.T) {
l := NewUnixSock(unixConfig)
l := NewUnixSock("t1", testUnixAddr)
require.Equal(t, "unix", l.Protocol())
}
func TestUnixSockInit(t *testing.T) {
l := NewUnixSock(unixConfig)
l := NewUnixSock("t1", testUnixAddr)
err := l.Init(logger)
l.Close(MockCloser)
require.NoError(t, err)
t2Config := unixConfig
t2Config.ID = "t2"
l2 := NewUnixSock(t2Config)
l2 := NewUnixSock("t2", testUnixAddr)
err = l2.Init(logger)
l2.Close(MockCloser)
require.NoError(t, err)
}
func TestUnixSockServeAndClose(t *testing.T) {
l := NewUnixSock(unixConfig)
l := NewUnixSock("t1", testUnixAddr)
err := l.Init(logger)
require.NoError(t, err)
@@ -80,7 +74,7 @@ func TestUnixSockServeAndClose(t *testing.T) {
}
func TestUnixSockEstablishThenEnd(t *testing.T) {
l := NewUnixSock(unixConfig)
l := NewUnixSock("t1", testUnixAddr)
err := l.Init(logger)
require.NoError(t, err)

View File

@@ -19,8 +19,6 @@ import (
"github.com/gorilla/websocket"
)
const TypeWS = "ws"
var (
// ErrInvalidMessage indicates that a message payload was not valid.
ErrInvalidMessage = errors.New("message type not binary")
@@ -31,7 +29,7 @@ type Websocket struct { // [MQTT-4.2.0-1]
sync.RWMutex
id string // the internal id of the listener
address string // the network address to bind to
config Config // configuration values for the listener
config *Config // configuration values for the listener
listen *http.Server // a http server for serving websocket connections
log *slog.Logger // server logger
establish EstablishFn // the server's establish connection handler
@@ -39,11 +37,15 @@ type Websocket struct { // [MQTT-4.2.0-1]
end uint32 // ensure the close methods are only called once
}
// NewWebsocket initializes and returns a new Websocket listener, listening on an address.
func NewWebsocket(config Config) *Websocket {
// NewWebsocket initialises and returns a new Websocket listener, listening on an address.
func NewWebsocket(id, address string, config *Config) *Websocket {
if config == nil {
config = new(Config)
}
return &Websocket{
id: config.ID,
address: config.Address,
id: id,
address: address,
config: config,
upgrader: &websocket.Upgrader{
Subprotocols: []string{"mqtt"},

View File

@@ -17,33 +17,35 @@ import (
)
func TestNewWebsocket(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
require.Equal(t, "t1", l.id)
require.Equal(t, testAddr, l.address)
}
func TestWebsocketID(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
require.Equal(t, "t1", l.ID())
}
func TestWebsocketAddress(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
require.Equal(t, testAddr, l.Address())
}
func TestWebsocketProtocol(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
require.Equal(t, "ws", l.Protocol())
}
func TestWebsocketProtocolTLS(t *testing.T) {
l := NewWebsocket(tlsConfig)
l := NewWebsocket("t1", testAddr, &Config{
TLSConfig: tlsConfigBasic,
})
require.Equal(t, "wss", l.Protocol())
}
func TestWebsocketInit(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
require.Nil(t, l.listen)
err := l.Init(logger)
require.NoError(t, err)
@@ -51,7 +53,7 @@ func TestWebsocketInit(t *testing.T) {
}
func TestWebsocketServeAndClose(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
_ = l.Init(logger)
o := make(chan bool)
@@ -72,7 +74,9 @@ func TestWebsocketServeAndClose(t *testing.T) {
}
func TestWebsocketServeTLSAndClose(t *testing.T) {
l := NewWebsocket(tlsConfig)
l := NewWebsocket("t1", testAddr, &Config{
TLSConfig: tlsConfigBasic,
})
err := l.Init(logger)
require.NoError(t, err)
@@ -92,9 +96,9 @@ func TestWebsocketServeTLSAndClose(t *testing.T) {
}
func TestWebsocketFailedToServe(t *testing.T) {
config := tlsConfig
config.Address = "wrong_addr"
l := NewWebsocket(config)
l := NewWebsocket("t1", "wrong_addr", &Config{
TLSConfig: tlsConfigBasic,
})
err := l.Init(logger)
require.NoError(t, err)
@@ -113,7 +117,7 @@ func TestWebsocketFailedToServe(t *testing.T) {
}
func TestWebsocketUpgrade(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
_ = l.Init(logger)
e := make(chan bool)
@@ -132,7 +136,7 @@ func TestWebsocketUpgrade(t *testing.T) {
}
func TestWebsocketConnectionReads(t *testing.T) {
l := NewWebsocket(basicConfig)
l := NewWebsocket("t1", testAddr, nil)
_ = l.Init(nil)
recv := make(chan []byte)

View File

@@ -1,81 +0,0 @@
package mempool
import (
"bytes"
"sync"
)
var bufPool = NewBuffer(0)
// GetBuffer takes a Buffer from the default buffer pool
func GetBuffer() *bytes.Buffer { return bufPool.Get() }
// PutBuffer returns Buffer to the default buffer pool
func PutBuffer(x *bytes.Buffer) { bufPool.Put(x) }
type BufferPool interface {
Get() *bytes.Buffer
Put(x *bytes.Buffer)
}
// NewBuffer returns a buffer pool. The max specify the max capacity of the Buffer the pool will
// return. If the Buffer becoomes large than max, it will no longer be returned to the pool. If
// max <= 0, no limit will be enforced.
func NewBuffer(max int) BufferPool {
if max > 0 {
return newBufferWithCap(max)
}
return newBuffer()
}
// Buffer is a Buffer pool.
type Buffer struct {
pool *sync.Pool
}
func newBuffer() *Buffer {
return &Buffer{
pool: &sync.Pool{
New: func() any { return new(bytes.Buffer) },
},
}
}
// Get a Buffer from the pool.
func (b *Buffer) Get() *bytes.Buffer {
return b.pool.Get().(*bytes.Buffer)
}
// Put the Buffer back into pool. It resets the Buffer for reuse.
func (b *Buffer) Put(x *bytes.Buffer) {
x.Reset()
b.pool.Put(x)
}
// BufferWithCap is a Buffer pool that
type BufferWithCap struct {
bp *Buffer
max int
}
func newBufferWithCap(max int) *BufferWithCap {
return &BufferWithCap{
bp: newBuffer(),
max: max,
}
}
// Get a Buffer from the pool.
func (b *BufferWithCap) Get() *bytes.Buffer {
return b.bp.Get()
}
// Put the Buffer back into the pool if the capacity doesn't exceed the limit. It resets the Buffer
// for reuse.
func (b *BufferWithCap) Put(x *bytes.Buffer) {
if x.Cap() > b.max {
return
}
b.bp.Put(x)
}

View File

@@ -1,96 +0,0 @@
package mempool
import (
"bytes"
"reflect"
"runtime/debug"
"testing"
"github.com/stretchr/testify/require"
)
func TestNewBuffer(t *testing.T) {
defer debug.SetGCPercent(debug.SetGCPercent(-1))
bp := NewBuffer(1000)
require.Equal(t, "*mempool.BufferWithCap", reflect.TypeOf(bp).String())
bp = NewBuffer(0)
require.Equal(t, "*mempool.Buffer", reflect.TypeOf(bp).String())
bp = NewBuffer(-1)
require.Equal(t, "*mempool.Buffer", reflect.TypeOf(bp).String())
}
func TestBuffer(t *testing.T) {
defer debug.SetGCPercent(debug.SetGCPercent(-1))
Size := 101
bp := NewBuffer(0)
buf := bp.Get()
for i := 0; i < Size; i++ {
buf.WriteByte('a')
}
bp.Put(buf)
buf = bp.Get()
require.Equal(t, 0, buf.Len())
}
func TestBufferWithCap(t *testing.T) {
defer debug.SetGCPercent(debug.SetGCPercent(-1))
Size := 101
bp := NewBuffer(100)
buf := bp.Get()
for i := 0; i < Size; i++ {
buf.WriteByte('a')
}
bp.Put(buf)
buf = bp.Get()
require.Equal(t, 0, buf.Len())
require.Equal(t, 0, buf.Cap())
}
func BenchmarkBufferPool(b *testing.B) {
bp := NewBuffer(0)
b.ResetTimer()
for i := 0; i < b.N; i++ {
b := bp.Get()
b.WriteString("this is a test")
bp.Put(b)
}
}
func BenchmarkBufferPoolWithCapLarger(b *testing.B) {
bp := NewBuffer(64 * 1024)
b.ResetTimer()
for i := 0; i < b.N; i++ {
b := bp.Get()
b.WriteString("this is a test")
bp.Put(b)
}
}
func BenchmarkBufferPoolWithCapLesser(b *testing.B) {
bp := NewBuffer(10)
b.ResetTimer()
for i := 0; i < b.N; i++ {
b := bp.Get()
b.WriteString("this is a test")
bp.Put(b)
}
}
func BenchmarkBufferWithoutPool(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
b := new(bytes.Buffer)
b.WriteString("this is a test")
_ = b
}
}

View File

@@ -12,8 +12,6 @@ import (
"strconv"
"strings"
"sync"
"github.com/mochi-mqtt/server/v2/mempool"
)
// All valid packet types and their packet identifiers.
@@ -300,8 +298,7 @@ func (s *Subscription) decode(b byte) {
// ConnectEncode encodes a connect packet.
func (pk *Packet) ConnectEncode(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.Write(encodeBytes(pk.Connect.ProtocolName))
nb.WriteByte(pk.ProtocolVersion)
@@ -318,8 +315,7 @@ func (pk *Packet) ConnectEncode(buf *bytes.Buffer) error {
nb.Write(encodeUint16(pk.Connect.Keepalive))
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
(&pk.Properties).Encode(pk.FixedHeader.Type, pk.Mods, pb, 0)
nb.Write(pb.Bytes())
}
@@ -328,8 +324,7 @@ func (pk *Packet) ConnectEncode(buf *bytes.Buffer) error {
if pk.Connect.WillFlag {
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
(&pk.Connect).WillProperties.Encode(WillProperties, pk.Mods, pb, 0)
nb.Write(pb.Bytes())
}
@@ -348,7 +343,7 @@ func (pk *Packet) ConnectEncode(buf *bytes.Buffer) error {
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -498,22 +493,19 @@ func (pk *Packet) ConnectValidate() Code {
// ConnackEncode encodes a Connack packet.
func (pk *Packet) ConnackEncode(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.WriteByte(encodeBool(pk.SessionPresent))
nb.WriteByte(pk.ReasonCode)
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len()+2) // +SessionPresent +ReasonCode
nb.Write(pb.Bytes())
}
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -544,21 +536,19 @@ func (pk *Packet) ConnackDecode(buf []byte) error {
// DisconnectEncode encodes a Disconnect packet.
func (pk *Packet) DisconnectEncode(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
if pk.ProtocolVersion == 5 {
nb.WriteByte(pk.ReasonCode)
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len())
nb.Write(pb.Bytes())
}
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -608,8 +598,7 @@ func (pk *Packet) PingrespDecode(buf []byte) error {
// PublishEncode encodes a Publish packet.
func (pk *Packet) PublishEncode(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.Write(encodeString(pk.TopicName)) // [MQTT-3.3.2-1]
@@ -621,16 +610,16 @@ func (pk *Packet) PublishEncode(buf *bytes.Buffer) error {
}
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len()+len(pk.Payload))
nb.Write(pb.Bytes())
}
pk.FixedHeader.Remaining = nb.Len() + len(pk.Payload)
nb.Write(pk.Payload)
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
buf.Write(pk.Payload)
_, _ = nb.WriteTo(buf)
return nil
}
@@ -701,13 +690,11 @@ func (pk *Packet) PublishValidate(topicAliasMaximum uint16) Code {
// encodePubAckRelRecComp encodes a Puback, Pubrel, Pubrec, or Pubcomp packet.
func (pk *Packet) encodePubAckRelRecComp(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.Write(encodeUint16(pk.PacketID))
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len())
if pk.ReasonCode >= ErrUnspecifiedError.Code || pb.Len() > 1 {
nb.WriteByte(pk.ReasonCode)
@@ -720,7 +707,7 @@ func (pk *Packet) encodePubAckRelRecComp(buf *bytes.Buffer) error {
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -844,13 +831,11 @@ func (pk *Packet) ReasonCodeValid() bool {
// SubackEncode encodes a Suback packet.
func (pk *Packet) SubackEncode(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.Write(encodeUint16(pk.PacketID))
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len()+len(pk.ReasonCodes))
nb.Write(pb.Bytes())
}
@@ -859,7 +844,7 @@ func (pk *Packet) SubackEncode(buf *bytes.Buffer) error {
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -893,12 +878,10 @@ func (pk *Packet) SubscribeEncode(buf *bytes.Buffer) error {
return ErrProtocolViolationNoPacketID
}
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.Write(encodeUint16(pk.PacketID))
xb := mempool.GetBuffer() // capture and write filters after length checks
defer mempool.PutBuffer(xb)
xb := bytes.NewBuffer([]byte{}) // capture and write filters after length checks
for _, opts := range pk.Filters {
xb.Write(encodeString(opts.Filter)) // [MQTT-3.8.3-1]
if pk.ProtocolVersion == 5 {
@@ -909,8 +892,7 @@ func (pk *Packet) SubscribeEncode(buf *bytes.Buffer) error {
}
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len()+xb.Len())
nb.Write(pb.Bytes())
}
@@ -919,7 +901,7 @@ func (pk *Packet) SubscribeEncode(buf *bytes.Buffer) error {
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -1001,21 +983,20 @@ func (pk *Packet) SubscribeValidate() Code {
// UnsubackEncode encodes an Unsuback packet.
func (pk *Packet) UnsubackEncode(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.Write(encodeUint16(pk.PacketID))
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len())
nb.Write(pb.Bytes())
nb.Write(pk.ReasonCodes)
}
nb.Write(pk.ReasonCodes)
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -1050,19 +1031,16 @@ func (pk *Packet) UnsubscribeEncode(buf *bytes.Buffer) error {
return ErrProtocolViolationNoPacketID
}
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.Write(encodeUint16(pk.PacketID))
xb := mempool.GetBuffer() // capture filters and write after length checks
defer mempool.PutBuffer(xb)
xb := bytes.NewBuffer([]byte{}) // capture filters and write after length checks
for _, sub := range pk.Filters {
xb.Write(encodeString(sub.Filter)) // [MQTT-3.10.3-1]
}
if pk.ProtocolVersion == 5 {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len()+xb.Len())
nb.Write(pb.Bytes())
}
@@ -1071,7 +1049,7 @@ func (pk *Packet) UnsubscribeEncode(buf *bytes.Buffer) error {
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}
@@ -1122,18 +1100,16 @@ func (pk *Packet) UnsubscribeValidate() Code {
// AuthEncode encodes an Auth packet.
func (pk *Packet) AuthEncode(buf *bytes.Buffer) error {
nb := mempool.GetBuffer()
defer mempool.PutBuffer(nb)
nb := bytes.NewBuffer([]byte{})
nb.WriteByte(pk.ReasonCode)
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
pk.Properties.Encode(pk.FixedHeader.Type, pk.Mods, pb, nb.Len())
nb.Write(pb.Bytes())
pk.FixedHeader.Remaining = nb.Len()
pk.FixedHeader.Encode(buf)
buf.Write(nb.Bytes())
_, _ = nb.WriteTo(buf)
return nil
}

View File

@@ -8,8 +8,6 @@ import (
"bytes"
"fmt"
"strings"
"github.com/mochi-mqtt/server/v2/mempool"
)
const (
@@ -201,8 +199,7 @@ func (p *Properties) Encode(pkt byte, mods Mods, b *bytes.Buffer, n int) {
return
}
buf := mempool.GetBuffer()
defer mempool.PutBuffer(buf)
var buf bytes.Buffer
if p.canEncode(pkt, PropPayloadFormat) && p.PayloadFormatFlag {
buf.WriteByte(PropPayloadFormat)
buf.WriteByte(p.PayloadFormat)
@@ -233,7 +230,7 @@ func (p *Properties) Encode(pkt byte, mods Mods, b *bytes.Buffer, n int) {
for _, v := range p.SubscriptionIdentifier {
if v > 0 {
buf.WriteByte(PropSubscriptionIdentifier)
encodeLength(buf, int64(v))
encodeLength(&buf, int64(v))
}
}
}
@@ -324,8 +321,7 @@ func (p *Properties) Encode(pkt byte, mods Mods, b *bytes.Buffer, n int) {
}
if !mods.DisallowProblemInfo && p.canEncode(pkt, PropUser) {
pb := mempool.GetBuffer()
defer mempool.PutBuffer(pb)
pb := bytes.NewBuffer([]byte{})
for _, v := range p.User {
pb.WriteByte(PropUser)
pb.Write(encodeString(v.Key))
@@ -359,7 +355,7 @@ func (p *Properties) Encode(pkt byte, mods Mods, b *bytes.Buffer, n int) {
}
encodeLength(b, int64(buf.Len()))
b.Write(buf.Bytes()) // [MQTT-3.1.3-10]
_, _ = buf.WriteTo(b) // [MQTT-3.1.3-10]
}
// Decode decodes property bytes into a properties struct.

257
server.go
View File

@@ -14,7 +14,6 @@ import (
"runtime"
"sort"
"strconv"
"strings"
"sync/atomic"
"time"
@@ -27,106 +26,91 @@ import (
)
const (
Version = "2.6.0" // the current server version.
Version = "2.4.1" // the current server version.
defaultSysTopicInterval int64 = 1 // the interval between $SYS topic publishes
LocalListener = "local"
InlineClientId = "inline"
)
var (
// Deprecated: Use NewDefaultServerCapabilities to avoid data race issue.
DefaultServerCapabilities = NewDefaultServerCapabilities()
// DefaultServerCapabilities defines the default features and capabilities provided by the server.
DefaultServerCapabilities = &Capabilities{
MaximumSessionExpiryInterval: math.MaxUint32, // maximum number of seconds to keep disconnected sessions
MaximumMessageExpiryInterval: 60 * 60 * 24, // maximum message expiry if message expiry is 0 or over
ReceiveMaximum: 1024, // maximum number of concurrent qos messages per client
MaximumQos: 2, // maximum qos value available to clients
RetainAvailable: 1, // retain messages is available
MaximumPacketSize: 0, // no maximum packet size
TopicAliasMaximum: math.MaxUint16, // maximum topic alias value
WildcardSubAvailable: 1, // wildcard subscriptions are available
SubIDAvailable: 1, // subscription identifiers are available
SharedSubAvailable: 1, // shared subscriptions are available
MinimumProtocolVersion: 3, // minimum supported mqtt version (3.0.0)
MaximumClientWritesPending: 1024 * 8, // maximum number of pending message writes for a client
}
ErrListenerIDExists = errors.New("listener id already exists") // a listener with the same id already exists
ErrConnectionClosed = errors.New("connection not open") // connection is closed
ErrInlineClientNotEnabled = errors.New("please set Options.InlineClient=true to use this feature") // inline client is not enabled by default
ErrOptionsUnreadable = errors.New("unable to read options from bytes")
)
// Capabilities indicates the capabilities and features provided by the server.
type Capabilities struct {
MaximumMessageExpiryInterval int64 `yaml:"maximum_message_expiry_interval" json:"maximum_message_expiry_interval"` // maximum message expiry if message expiry is 0 or over
MaximumClientWritesPending int32 `yaml:"maximum_client_writes_pending" json:"maximum_client_writes_pending"` // maximum number of pending message writes for a client
MaximumSessionExpiryInterval uint32 `yaml:"maximum_session_expiry_interval" json:"maximum_session_expiry_interval"` // maximum number of seconds to keep disconnected sessions
MaximumPacketSize uint32 `yaml:"maximum_packet_size" json:"maximum_packet_size"` // maximum packet size, no limit if 0
maximumPacketID uint32 // unexported, used for testing only
ReceiveMaximum uint16 `yaml:"receive_maximum" json:"receive_maximum"` // maximum number of concurrent qos messages per client
MaximumInflight uint16 `yaml:"maximum_inflight" json:"maximum_inflight"` // maximum number of qos > 0 messages can be stored, 0(=8192)-65535
TopicAliasMaximum uint16 `yaml:"topic_alias_maximum" json:"topic_alias_maximum"` // maximum topic alias value
SharedSubAvailable byte `yaml:"shared_sub_available" json:"shared_sub_available"` // support of shared subscriptions
MinimumProtocolVersion byte `yaml:"minimum_protocol_version" json:"minimum_protocol_version"` // minimum supported mqtt version
Compatibilities Compatibilities `yaml:"compatibilities" json:"compatibilities"` // version compatibilities the server provides
MaximumQos byte `yaml:"maximum_qos" json:"maximum_qos"` // maximum qos value available to clients
RetainAvailable byte `yaml:"retain_available" json:"retain_available"` // support of retain messages
WildcardSubAvailable byte `yaml:"wildcard_sub_available" json:"wildcard_sub_available"` // support of wildcard subscriptions
SubIDAvailable byte `yaml:"sub_id_available" json:"sub_id_available"` // support of subscription identifiers
}
// NewDefaultServerCapabilities defines the default features and capabilities provided by the server.
func NewDefaultServerCapabilities() *Capabilities {
return &Capabilities{
MaximumMessageExpiryInterval: 60 * 60 * 24, // maximum message expiry if message expiry is 0 or over
MaximumClientWritesPending: 1024 * 8, // maximum number of pending message writes for a client
MaximumSessionExpiryInterval: math.MaxUint32, // maximum number of seconds to keep disconnected sessions
MaximumPacketSize: 0, // no maximum packet size
maximumPacketID: math.MaxUint16,
ReceiveMaximum: 1024, // maximum number of concurrent qos messages per client
MaximumInflight: 1024 * 8, // maximum number of qos > 0 messages can be stored
TopicAliasMaximum: math.MaxUint16, // maximum topic alias value
SharedSubAvailable: 1, // shared subscriptions are available
MinimumProtocolVersion: 3, // minimum supported mqtt version (3.0.0)
MaximumQos: 2, // maximum qos value available to clients
RetainAvailable: 1, // retain messages is available
WildcardSubAvailable: 1, // wildcard subscriptions are available
SubIDAvailable: 1, // subscription identifiers are available
}
MaximumMessageExpiryInterval int64
MaximumClientWritesPending int32
MaximumSessionExpiryInterval uint32
MaximumPacketSize uint32
maximumPacketID uint32 // unexported, used for testing only
ReceiveMaximum uint16
TopicAliasMaximum uint16
SharedSubAvailable byte
MinimumProtocolVersion byte
Compatibilities Compatibilities
MaximumQos byte
RetainAvailable byte
WildcardSubAvailable byte
SubIDAvailable byte
}
// Compatibilities provides flags for using compatibility modes.
type Compatibilities struct {
ObscureNotAuthorized bool `yaml:"obscure_not_authorized" json:"obscure_not_authorized"` // return unspecified errors instead of not authorized
PassiveClientDisconnect bool `yaml:"passive_client_disconnect" json:"passive_client_disconnect"` // don't disconnect the client forcefully after sending disconnect packet (paho - spec violation)
AlwaysReturnResponseInfo bool `yaml:"always_return_response_info" json:"always_return_response_info"` // always return response info (useful for testing)
RestoreSysInfoOnRestart bool `yaml:"restore_sys_info_on_restart" json:"restore_sys_info_on_restart"` // restore system info from store as if server never stopped
NoInheritedPropertiesOnAck bool `yaml:"no_inherited_properties_on_ack" json:"no_inherited_properties_on_ack"` // don't allow inherited user properties on ack (paho - spec violation)
ObscureNotAuthorized bool // return unspecified errors instead of not authorized
PassiveClientDisconnect bool // don't disconnect the client forcefully after sending disconnect packet (paho - spec violation)
AlwaysReturnResponseInfo bool // always return response info (useful for testing)
RestoreSysInfoOnRestart bool // restore system info from store as if server never stopped
NoInheritedPropertiesOnAck bool // don't allow inherited user properties on ack (paho - spec violation)
}
// Options contains configurable options for the server.
type Options struct {
// Listeners specifies any listeners which should be dynamically added on serve. Used when setting listeners by config.
Listeners []listeners.Config `yaml:"listeners" json:"listeners"`
// Hooks specifies any hooks which should be dynamically added on serve. Used when setting hooks by config.
Hooks []HookLoadConfig `yaml:"hooks" json:"hooks"`
// Capabilities defines the server features and behaviour. If you only wish to modify
// several of these values, set them explicitly - e.g.
// server.Options.Capabilities.MaximumClientWritesPending = 16 * 1024
Capabilities *Capabilities `yaml:"capabilities" json:"capabilities"`
Capabilities *Capabilities
// ClientNetWriteBufferSize specifies the size of the client *bufio.Writer write buffer.
ClientNetWriteBufferSize int `yaml:"client_net_write_buffer_size" json:"client_net_write_buffer_size"`
ClientNetWriteBufferSize int
// ClientNetReadBufferSize specifies the size of the client *bufio.Reader read buffer.
ClientNetReadBufferSize int `yaml:"client_net_read_buffer_size" json:"client_net_read_buffer_size"`
ClientNetReadBufferSize int
// Logger specifies a custom configured implementation of zerolog to override
// the servers default logger configuration. If you wish to change the log level,
// of the default logger, you can do so by setting:
// server := mqtt.New(nil)
// of the default logger, you can do so by setting
// server := mqtt.New(nil)
// level := new(slog.LevelVar)
// server.Slog = slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
// Level: level,
// }))
// level.Set(slog.LevelDebug)
Logger *slog.Logger `yaml:"-" json:"-"`
Logger *slog.Logger
// SysTopicResendInterval specifies the interval between $SYS topic updates in seconds.
SysTopicResendInterval int64 `yaml:"sys_topic_resend_interval" json:"sys_topic_resend_interval"`
SysTopicResendInterval int64
// Enable Inline client to allow direct subscribing and publishing from the parent codebase,
// with negligible performance difference (disabled by default to prevent confusion in statistics).
InlineClient bool `yaml:"inline_client" json:"inline_client"`
InlineClient bool
}
// Server is an MQTT broker server. It should be created with server.New()
@@ -206,15 +190,11 @@ func New(opts *Options) *Server {
// ensureDefaults ensures that the server starts with sane default values, if none are provided.
func (o *Options) ensureDefaults() {
if o.Capabilities == nil {
o.Capabilities = NewDefaultServerCapabilities()
o.Capabilities = DefaultServerCapabilities
}
o.Capabilities.maximumPacketID = math.MaxUint16 // spec maximum is 65535
if o.Capabilities.MaximumInflight == 0 {
o.Capabilities.MaximumInflight = 1024 * 8
}
if o.SysTopicResendInterval == 0 {
o.SysTopicResendInterval = defaultSysTopicInterval
}
@@ -253,6 +233,8 @@ func (s *Server) NewClient(c net.Conn, listener string, id string, inline bool)
// By default, we don't want to restrict developer publishes,
// but if you do, reset this after creating inline client.
cl.State.Inflight.ResetReceiveQuota(math.MaxInt32)
} else {
go cl.WriteLoop() // can only write to real clients
}
return cl
@@ -270,17 +252,6 @@ func (s *Server) AddHook(hook Hook, config any) error {
return s.hooks.Add(hook, config)
}
// AddHooksFromConfig adds hooks to the server which were specified in the hooks config (usually from a config file).
// New built-in hooks should be added to this list.
func (s *Server) AddHooksFromConfig(hooks []HookLoadConfig) error {
for _, h := range hooks {
if err := s.AddHook(h.Hook, h.Config); err != nil {
return err
}
}
return nil
}
// AddListener adds a new network listener to the server, for receiving incoming client connections.
func (s *Server) AddListener(l listeners.Listener) error {
if _, ok := s.Listeners.Get(l.ID()); ok {
@@ -299,55 +270,12 @@ func (s *Server) AddListener(l listeners.Listener) error {
return nil
}
// AddListenersFromConfig adds listeners to the server which were specified in the listeners config (usually from a config file).
// New built-in listeners should be added to this list.
func (s *Server) AddListenersFromConfig(configs []listeners.Config) error {
for _, conf := range configs {
var l listeners.Listener
switch strings.ToLower(conf.Type) {
case listeners.TypeTCP:
l = listeners.NewTCP(conf)
case listeners.TypeWS:
l = listeners.NewWebsocket(conf)
case listeners.TypeUnix:
l = listeners.NewUnixSock(conf)
case listeners.TypeHealthCheck:
l = listeners.NewHTTPHealthCheck(conf)
case listeners.TypeSysInfo:
l = listeners.NewHTTPStats(conf, s.Info)
case listeners.TypeMock:
l = listeners.NewMockListener(conf.ID, conf.Address)
default:
s.Log.Error("listener type unavailable by config", "listener", conf.Type)
continue
}
if err := s.AddListener(l); err != nil {
return err
}
}
return nil
}
// Serve starts the event loops responsible for establishing client connections
// on all attached listeners, publishing the system topics, and starting all hooks.
func (s *Server) Serve() error {
s.Log.Info("mochi mqtt starting", "version", Version)
defer s.Log.Info("mochi mqtt server started")
if len(s.Options.Listeners) > 0 {
err := s.AddListenersFromConfig(s.Options.Listeners)
if err != nil {
return err
}
}
if len(s.Options.Hooks) > 0 {
err := s.AddHooksFromConfig(s.Options.Hooks)
if err != nil {
return err
}
}
if s.hooks.Provides(
StoredClients,
StoredInflightMessages,
@@ -404,8 +332,6 @@ func (s *Server) EstablishConnection(listener string, c net.Conn) error {
func (s *Server) attachClient(cl *Client, listener string) error {
defer s.Listeners.ClientsWg.Done()
s.Listeners.ClientsWg.Add(1)
go cl.WriteLoop()
defer cl.Stop(nil)
pk, err := s.readConnectionPacket(cl)
@@ -474,7 +400,7 @@ func (s *Server) attachClient(cl *Client, listener string) error {
s.hooks.OnDisconnect(cl, err, expire)
if expire && atomic.LoadUint32(&cl.State.isTakenOver) == 0 {
cl.ClearInflights()
cl.ClearInflights(math.MaxInt64, 0)
s.UnsubscribeClient(cl)
s.Clients.Delete(cl.ID) // [MQTT-4.1.0-2] ![MQTT-3.1.2-23]
}
@@ -552,7 +478,7 @@ func (s *Server) inheritClientSession(pk packets.Packet, cl *Client) bool {
_ = s.DisconnectClient(existing, packets.ErrSessionTakenOver) // [MQTT-3.1.4-3]
if pk.Connect.Clean || (existing.Properties.Clean && existing.Properties.ProtocolVersion < 5) { // [MQTT-3.1.2-4] [MQTT-3.1.4-4]
s.UnsubscribeClient(existing)
existing.ClearInflights()
existing.ClearInflights(math.MaxInt64, 0)
atomic.StoreUint32(&existing.State.isTakenOver, 1) // only set isTakenOver after unsubscribe has occurred
return false // [MQTT-3.2.2-3]
}
@@ -577,7 +503,7 @@ func (s *Server) inheritClientSession(pk packets.Packet, cl *Client) bool {
// Clean the state of the existing client to prevent sequential take-overs
// from increasing memory usage by inflights + subs * client-id.
s.UnsubscribeClient(existing)
existing.ClearInflights()
existing.ClearInflights(math.MaxInt64, 0)
s.Log.Debug("session taken over", "client", cl.ID, "old_remote", existing.Net.Remote, "new_remote", cl.Net.Remote)
@@ -1043,17 +969,9 @@ func (s *Server) publishToClient(cl *Client, sub packets.Subscription, pk packet
}
if out.FixedHeader.Qos > 0 {
if cl.State.Inflight.Len() >= int(s.Options.Capabilities.MaximumInflight) {
// add hook?
atomic.AddInt64(&s.Info.InflightDropped, 1)
s.Log.Warn("client store quota reached", "client", cl.ID, "listener", cl.Net.Listener)
return out, packets.ErrQuotaExceeded
}
i, err := cl.NextPacketID() // [MQTT-4.3.2-1] [MQTT-4.3.3-1]
if err != nil {
s.hooks.OnPacketIDExhausted(cl, pk)
atomic.AddInt64(&s.Info.InflightDropped, 1)
s.Log.Warn("packet ids exhausted", "error", err, "client", cl.ID, "listener", cl.Net.Listener)
return out, packets.ErrQuotaExceeded
}
@@ -1084,10 +1002,8 @@ func (s *Server) publishToClient(cl *Client, sub packets.Subscription, pk packet
default:
atomic.AddInt64(&s.Info.MessagesDropped, 1)
cl.ops.hooks.OnPublishDropped(cl, pk)
if out.FixedHeader.Qos > 0 {
cl.State.Inflight.Delete(out.PacketID) // packet was dropped due to irregular circumstances, so rollback inflight.
cl.State.Inflight.IncreaseSendQuota()
}
cl.State.Inflight.Delete(out.PacketID) // packet was dropped due to irregular circumstances, so rollback inflight.
cl.State.Inflight.IncreaseSendQuota()
return out, packets.ErrPendingClientWritesExceeded
}
@@ -1435,28 +1351,27 @@ func (s *Server) publishSysTopics() {
atomic.StoreInt64(&s.Info.ClientsTotal, int64(s.Clients.Len()))
atomic.StoreInt64(&s.Info.ClientsDisconnected, atomic.LoadInt64(&s.Info.ClientsTotal)-atomic.LoadInt64(&s.Info.ClientsConnected))
info := s.Info.Clone()
topics := map[string]string{
SysPrefix + "/broker/version": s.Info.Version,
SysPrefix + "/broker/time": Int64toa(info.Time),
SysPrefix + "/broker/uptime": Int64toa(info.Uptime),
SysPrefix + "/broker/started": Int64toa(info.Started),
SysPrefix + "/broker/load/bytes/received": Int64toa(info.BytesReceived),
SysPrefix + "/broker/load/bytes/sent": Int64toa(info.BytesSent),
SysPrefix + "/broker/clients/connected": Int64toa(info.ClientsConnected),
SysPrefix + "/broker/clients/disconnected": Int64toa(info.ClientsDisconnected),
SysPrefix + "/broker/clients/maximum": Int64toa(info.ClientsMaximum),
SysPrefix + "/broker/clients/total": Int64toa(info.ClientsTotal),
SysPrefix + "/broker/packets/received": Int64toa(info.PacketsReceived),
SysPrefix + "/broker/packets/sent": Int64toa(info.PacketsSent),
SysPrefix + "/broker/messages/received": Int64toa(info.MessagesReceived),
SysPrefix + "/broker/messages/sent": Int64toa(info.MessagesSent),
SysPrefix + "/broker/messages/dropped": Int64toa(info.MessagesDropped),
SysPrefix + "/broker/messages/inflight": Int64toa(info.Inflight),
SysPrefix + "/broker/retained": Int64toa(info.Retained),
SysPrefix + "/broker/subscriptions": Int64toa(info.Subscriptions),
SysPrefix + "/broker/system/memory": Int64toa(info.MemoryAlloc),
SysPrefix + "/broker/system/threads": Int64toa(info.Threads),
SysPrefix + "/broker/time": AtomicItoa(&s.Info.Time),
SysPrefix + "/broker/uptime": AtomicItoa(&s.Info.Uptime),
SysPrefix + "/broker/started": AtomicItoa(&s.Info.Started),
SysPrefix + "/broker/load/bytes/received": AtomicItoa(&s.Info.BytesReceived),
SysPrefix + "/broker/load/bytes/sent": AtomicItoa(&s.Info.BytesSent),
SysPrefix + "/broker/clients/connected": AtomicItoa(&s.Info.ClientsConnected),
SysPrefix + "/broker/clients/disconnected": AtomicItoa(&s.Info.ClientsDisconnected),
SysPrefix + "/broker/clients/maximum": AtomicItoa(&s.Info.ClientsMaximum),
SysPrefix + "/broker/clients/total": AtomicItoa(&s.Info.ClientsTotal),
SysPrefix + "/broker/packets/received": AtomicItoa(&s.Info.PacketsReceived),
SysPrefix + "/broker/packets/sent": AtomicItoa(&s.Info.PacketsSent),
SysPrefix + "/broker/messages/received": AtomicItoa(&s.Info.MessagesReceived),
SysPrefix + "/broker/messages/sent": AtomicItoa(&s.Info.MessagesSent),
SysPrefix + "/broker/messages/dropped": AtomicItoa(&s.Info.MessagesDropped),
SysPrefix + "/broker/messages/inflight": AtomicItoa(&s.Info.Inflight),
SysPrefix + "/broker/retained": AtomicItoa(&s.Info.Retained),
SysPrefix + "/broker/subscriptions": AtomicItoa(&s.Info.Subscriptions),
SysPrefix + "/broker/system/memory": AtomicItoa(&s.Info.MemoryAlloc),
SysPrefix + "/broker/system/threads": AtomicItoa(&s.Info.Threads),
}
for topic, payload := range topics {
@@ -1466,7 +1381,7 @@ func (s *Server) publishSysTopics() {
s.publishToSubscribers(pk)
}
s.hooks.OnSysInfoTick(info)
s.hooks.OnSysInfoTick(s.Info)
}
// Close attempts to gracefully shut down the server, all listeners, clients, and stores.
@@ -1638,18 +1553,7 @@ func (s *Server) loadClients(v []storage.Client) {
MaximumPacketSize: c.Properties.MaximumPacketSize,
}
cl.Properties.Will = Will(c.Will)
// cancel the context, update cl.State such as disconnected time and stopCause.
cl.Stop(packets.ErrServerShuttingDown)
expire := (cl.Properties.ProtocolVersion == 5 && cl.Properties.Props.SessionExpiryInterval == 0) || (cl.Properties.ProtocolVersion < 5 && cl.Properties.Clean)
s.hooks.OnDisconnect(cl, packets.ErrServerShuttingDown, expire)
if expire {
cl.ClearInflights()
s.UnsubscribeClient(cl)
} else {
s.Clients.Add(cl)
}
s.Clients.Add(cl)
}
}
@@ -1693,14 +1597,7 @@ func (s *Server) clearExpiredClients(dt int64) {
// clearExpiredRetainedMessage deletes retained messages from topics if they have expired.
func (s *Server) clearExpiredRetainedMessages(now int64) {
for filter, pk := range s.Topics.Retained.GetAll() {
expired := pk.ProtocolVersion == 5 && pk.Expiry > 0 && pk.Expiry < now // [MQTT-3.3.2-5]
// If the maximum message expiry interval is set (greater than 0), and the message
// retention period exceeds the maximum expiry, the message will be forcibly removed.
enforced := s.Options.Capabilities.MaximumMessageExpiryInterval > 0 &&
now-pk.Created > s.Options.Capabilities.MaximumMessageExpiryInterval
if expired || enforced {
if (pk.Expiry > 0 && pk.Expiry < now) || pk.Created+s.Options.Capabilities.MaximumMessageExpiryInterval < now {
s.Topics.Retained.Delete(filter)
s.hooks.OnRetainedExpired(filter)
}
@@ -1710,7 +1607,7 @@ func (s *Server) clearExpiredRetainedMessages(now int64) {
// clearExpiredInflights deletes any inflight messages which have expired.
func (s *Server) clearExpiredInflights(now int64) {
for _, client := range s.Clients.GetAll() {
if deleted := client.ClearExpiredInflights(now, s.Options.Capabilities.MaximumMessageExpiryInterval); len(deleted) > 0 {
if deleted := client.ClearInflights(now, s.Options.Capabilities.MaximumMessageExpiryInterval); len(deleted) > 0 {
for _, id := range deleted {
s.hooks.OnQosDropped(client, packets.Packet{PacketID: id})
}
@@ -1735,7 +1632,7 @@ func (s *Server) sendDelayedLWT(dt int64) {
}
}
// Int64toa converts an int64 to a string.
func Int64toa(v int64) string {
return strconv.FormatInt(v, 10)
// AtomicItoa converts an int64 point to a string.
func AtomicItoa(ptr *int64) string {
return strconv.FormatInt(atomic.LoadInt64(ptr), 10)
}

View File

@@ -96,24 +96,24 @@ func (h *DelayHook) OnDisconnect(cl *Client, err error, expire bool) {
}
func newServer() *Server {
cc := NewDefaultServerCapabilities()
cc := *DefaultServerCapabilities
cc.MaximumMessageExpiryInterval = 0
cc.ReceiveMaximum = 0
s := New(&Options{
Logger: logger,
Capabilities: cc,
Capabilities: &cc,
})
_ = s.AddHook(new(AllowHook), nil)
return s
}
func newServerWithInlineClient() *Server {
cc := NewDefaultServerCapabilities()
cc := *DefaultServerCapabilities
cc.MaximumMessageExpiryInterval = 0
cc.ReceiveMaximum = 0
s := New(&Options{
Logger: logger,
Capabilities: cc,
Capabilities: &cc,
InlineClient: true,
})
_ = s.AddHook(new(AllowHook), nil)
@@ -125,7 +125,7 @@ func TestOptionsSetDefaults(t *testing.T) {
opts.ensureDefaults()
require.Equal(t, defaultSysTopicInterval, opts.SysTopicResendInterval)
require.Equal(t, NewDefaultServerCapabilities(), opts.Capabilities)
require.Equal(t, DefaultServerCapabilities, opts.Capabilities)
opts = new(Options)
opts.ensureDefaults()
@@ -220,34 +220,6 @@ func TestServerAddListener(t *testing.T) {
require.Equal(t, ErrListenerIDExists, err)
}
func TestServerAddHooksFromConfig(t *testing.T) {
s := newServer()
defer s.Close()
require.NotNil(t, s)
s.Log = logger
hooks := []HookLoadConfig{
{Hook: new(modifiedHookBase)},
}
err := s.AddHooksFromConfig(hooks)
require.NoError(t, err)
}
func TestServerAddHooksFromConfigError(t *testing.T) {
s := newServer()
defer s.Close()
require.NotNil(t, s)
s.Log = logger
hooks := []HookLoadConfig{
{Hook: new(modifiedHookBase), Config: map[string]interface{}{}},
}
err := s.AddHooksFromConfig(hooks)
require.Error(t, err)
}
func TestServerAddListenerInitFailure(t *testing.T) {
s := newServer()
defer s.Close()
@@ -260,60 +232,6 @@ func TestServerAddListenerInitFailure(t *testing.T) {
require.Error(t, err)
}
func TestServerAddListenersFromConfig(t *testing.T) {
s := newServer()
defer s.Close()
require.NotNil(t, s)
s.Log = logger
lc := []listeners.Config{
{Type: listeners.TypeTCP, ID: "tcp", Address: ":1883"},
{Type: listeners.TypeWS, ID: "ws", Address: ":1882"},
{Type: listeners.TypeHealthCheck, ID: "health", Address: ":1881"},
{Type: listeners.TypeSysInfo, ID: "info", Address: ":1880"},
{Type: listeners.TypeUnix, ID: "unix", Address: "mochi.sock"},
{Type: listeners.TypeMock, ID: "mock", Address: "0"},
{Type: "unknown", ID: "unknown"},
}
err := s.AddListenersFromConfig(lc)
require.NoError(t, err)
require.Equal(t, 6, s.Listeners.Len())
tcp, _ := s.Listeners.Get("tcp")
require.Equal(t, "[::]:1883", tcp.Address())
ws, _ := s.Listeners.Get("ws")
require.Equal(t, ":1882", ws.Address())
health, _ := s.Listeners.Get("health")
require.Equal(t, ":1881", health.Address())
info, _ := s.Listeners.Get("info")
require.Equal(t, ":1880", info.Address())
unix, _ := s.Listeners.Get("unix")
require.Equal(t, "mochi.sock", unix.Address())
mock, _ := s.Listeners.Get("mock")
require.Equal(t, "0", mock.Address())
}
func TestServerAddListenersFromConfigError(t *testing.T) {
s := newServer()
defer s.Close()
require.NotNil(t, s)
s.Log = logger
lc := []listeners.Config{
{Type: listeners.TypeTCP, ID: "tcp", Address: "x"},
}
err := s.AddListenersFromConfig(lc)
require.Error(t, err)
require.Equal(t, 0, s.Listeners.Len())
}
func TestServerServe(t *testing.T) {
s := newServer()
defer s.Close()
@@ -335,57 +253,6 @@ func TestServerServe(t *testing.T) {
require.Equal(t, true, listener.(*listeners.MockListener).IsServing())
}
func TestServerServeFromConfig(t *testing.T) {
s := newServer()
defer s.Close()
require.NotNil(t, s)
s.Options.Listeners = []listeners.Config{
{Type: listeners.TypeMock, ID: "mock", Address: "0"},
}
s.Options.Hooks = []HookLoadConfig{
{Hook: new(modifiedHookBase)},
}
err := s.Serve()
require.NoError(t, err)
time.Sleep(time.Millisecond)
require.Equal(t, 1, s.Listeners.Len())
listener, ok := s.Listeners.Get("mock")
require.Equal(t, true, ok)
require.Equal(t, true, listener.(*listeners.MockListener).IsServing())
}
func TestServerServeFromConfigListenerError(t *testing.T) {
s := newServer()
defer s.Close()
require.NotNil(t, s)
s.Options.Listeners = []listeners.Config{
{Type: listeners.TypeTCP, ID: "tcp", Address: "x"},
}
err := s.Serve()
require.Error(t, err)
}
func TestServerServeFromConfigHookError(t *testing.T) {
s := newServer()
defer s.Close()
require.NotNil(t, s)
s.Options.Hooks = []HookLoadConfig{
{Hook: new(modifiedHookBase), Config: map[string]interface{}{}},
}
err := s.Serve()
require.Error(t, err)
}
func TestServerServeReadStoreFailure(t *testing.T) {
s := newServer()
defer s.Close()
@@ -1662,10 +1529,10 @@ func TestServerProcessPublishACLCheckDeny(t *testing.T) {
for _, tx := range tt {
t.Run(tx.name, func(t *testing.T) {
cc := NewDefaultServerCapabilities()
cc := *DefaultServerCapabilities
s := New(&Options{
Logger: logger,
Capabilities: cc,
Capabilities: &cc,
})
_ = s.AddHook(new(DenyHook), nil)
_ = s.Serve()
@@ -2040,7 +1907,6 @@ func TestPublishToClientSubscriptionDowngradeQos(t *testing.T) {
}
func TestPublishToClientExceedClientWritesPending(t *testing.T) {
var sendQuota uint16 = 5
s := newServer()
_, w := net.Pipe()
@@ -2051,12 +1917,9 @@ func TestPublishToClientExceedClientWritesPending(t *testing.T) {
options: &Options{
Capabilities: &Capabilities{
MaximumClientWritesPending: 3,
maximumPacketID: 10,
},
},
})
cl.Properties.Props.ReceiveMaximum = sendQuota
cl.State.Inflight.ResetSendQuota(int32(cl.Properties.Props.ReceiveMaximum))
s.Clients.Add(cl)
@@ -2065,20 +1928,9 @@ func TestPublishToClientExceedClientWritesPending(t *testing.T) {
atomic.AddInt32(&cl.State.outboundQty, 1)
}
id, _ := cl.NextPacketID()
cl.State.Inflight.Set(packets.Packet{PacketID: uint16(id)})
cl.State.Inflight.DecreaseSendQuota()
sendQuota--
_, err := s.publishToClient(cl, packets.Subscription{Filter: "a/b/c", Qos: 2}, packets.Packet{})
require.Error(t, err)
require.ErrorIs(t, packets.ErrPendingClientWritesExceeded, err)
require.Equal(t, int32(sendQuota), atomic.LoadInt32(&cl.State.Inflight.sendQuota))
_, err = s.publishToClient(cl, packets.Subscription{Filter: "a/b/c", Qos: 2}, packets.Packet{FixedHeader: packets.FixedHeader{Qos: 1}})
require.Error(t, err)
require.ErrorIs(t, packets.ErrPendingClientWritesExceeded, err)
require.Equal(t, int32(sendQuota), atomic.LoadInt32(&cl.State.Inflight.sendQuota))
}
func TestPublishToClientServerTopicAlias(t *testing.T) {
@@ -2134,22 +1986,6 @@ func TestPublishToClientMqtt5RetainAsPublishedTrueLeverageNoConn(t *testing.T) {
require.ErrorIs(t, err, packets.CodeDisconnect)
}
func TestPublishToClientExceedMaximumInflight(t *testing.T) {
const MaxInflight uint16 = 5
s := newServer()
cl, _, _ := newTestClient()
s.Options.Capabilities.MaximumInflight = MaxInflight
cl.ops.options.Capabilities.MaximumInflight = MaxInflight
for i := uint16(0); i < MaxInflight; i++ {
cl.State.Inflight.Set(packets.Packet{PacketID: i})
}
_, err := s.publishToClient(cl, packets.Subscription{Filter: "a/b/c", Qos: 1}, *packets.TPacketData[packets.Publish].Get(packets.TPublishQos1).Packet)
require.Error(t, err)
require.ErrorIs(t, err, packets.ErrQuotaExceeded)
require.Equal(t, int64(1), atomic.LoadInt64(&s.Info.InflightDropped))
}
func TestPublishToClientExhaustedPacketID(t *testing.T) {
s := newServer()
cl, _, _ := newTestClient()
@@ -2160,7 +1996,6 @@ func TestPublishToClientExhaustedPacketID(t *testing.T) {
_, err := s.publishToClient(cl, packets.Subscription{Filter: "a/b/c", Qos: 1}, *packets.TPacketData[packets.Publish].Get(packets.TPublishQos1).Packet)
require.Error(t, err)
require.ErrorIs(t, err, packets.ErrQuotaExceeded)
require.Equal(t, int64(1), atomic.LoadInt64(&s.Info.InflightDropped))
}
func TestPublishToClientACLNotAuthorized(t *testing.T) {
@@ -3293,50 +3128,15 @@ func TestServerLoadClients(t *testing.T) {
{ID: "mochi"},
{ID: "zen"},
{ID: "mochi-co"},
{ID: "v3-clean", ProtocolVersion: 4, Clean: true},
{ID: "v3-not-clean", ProtocolVersion: 4, Clean: false},
{
ID: "v5-clean",
ProtocolVersion: 5,
Clean: true,
Properties: storage.ClientProperties{
SessionExpiryInterval: 10,
},
},
{
ID: "v5-expire-interval-0",
ProtocolVersion: 5,
Properties: storage.ClientProperties{
SessionExpiryInterval: 0,
},
},
{
ID: "v5-expire-interval-not-0",
ProtocolVersion: 5,
Properties: storage.ClientProperties{
SessionExpiryInterval: 10,
},
},
}
s := newServer()
require.Equal(t, 0, s.Clients.Len())
s.loadClients(v)
require.Equal(t, 6, s.Clients.Len())
require.Equal(t, 3, s.Clients.Len())
cl, ok := s.Clients.Get("mochi")
require.True(t, ok)
require.Equal(t, "mochi", cl.ID)
_, ok = s.Clients.Get("v3-clean")
require.False(t, ok)
_, ok = s.Clients.Get("v3-not-clean")
require.True(t, ok)
_, ok = s.Clients.Get("v5-clean")
require.True(t, ok)
_, ok = s.Clients.Get("v5-expire-interval-0")
require.False(t, ok)
_, ok = s.Clients.Get("v5-expire-interval-not-0")
require.True(t, ok)
}
func TestServerLoadSubscriptions(t *testing.T) {
@@ -3459,11 +3259,6 @@ func TestServerClearExpiredInflights(t *testing.T) {
s.clearExpiredInflights(n)
require.Len(t, cl.State.Inflight.GetAll(false), 2)
require.Equal(t, int64(-3), s.Info.Inflight)
s.Options.Capabilities.MaximumMessageExpiryInterval = 0
cl.State.Inflight.Set(packets.Packet{PacketID: 8, Expiry: n - 8})
s.clearExpiredInflights(n)
require.Len(t, cl.State.Inflight.GetAll(false), 3)
}
func TestServerClearExpiredRetained(t *testing.T) {
@@ -3472,28 +3267,15 @@ func TestServerClearExpiredRetained(t *testing.T) {
s.Options.Capabilities.MaximumMessageExpiryInterval = 4
n := time.Now().Unix()
s.Topics.Retained.Add("a/b/c", packets.Packet{ProtocolVersion: 5, Created: n, Expiry: n - 1})
s.Topics.Retained.Add("d/e/f", packets.Packet{ProtocolVersion: 5, Created: n, Expiry: n - 2})
s.Topics.Retained.Add("g/h/i", packets.Packet{ProtocolVersion: 5, Created: n - 3}) // within bounds
s.Topics.Retained.Add("j/k/l", packets.Packet{ProtocolVersion: 5, Created: n - 5}) // over max server expiry limit
s.Topics.Retained.Add("m/n/o", packets.Packet{ProtocolVersion: 5, Created: n})
s.Topics.Retained.Add("a/b/c", packets.Packet{Created: n, Expiry: n - 1})
s.Topics.Retained.Add("d/e/f", packets.Packet{Created: n, Expiry: n - 2})
s.Topics.Retained.Add("g/h/i", packets.Packet{Created: n - 3}) // within bounds
s.Topics.Retained.Add("j/k/l", packets.Packet{Created: n - 5}) // over max server expiry limit
s.Topics.Retained.Add("m/n/o", packets.Packet{Created: n})
require.Len(t, s.Topics.Retained.GetAll(), 5)
s.clearExpiredRetainedMessages(n)
require.Len(t, s.Topics.Retained.GetAll(), 2)
s.Topics.Retained.Add("p/q/r", packets.Packet{Created: n, Expiry: n - 1})
s.Topics.Retained.Add("s/t/u", packets.Packet{Created: n, Expiry: n - 2}) // expiry is ineffective for v3.
s.Topics.Retained.Add("v/w/x", packets.Packet{Created: n - 3}) // within bounds for v3
s.Topics.Retained.Add("y/z/1", packets.Packet{Created: n - 5}) // over max server expiry limit
require.Len(t, s.Topics.Retained.GetAll(), 6)
s.clearExpiredRetainedMessages(n)
require.Len(t, s.Topics.Retained.GetAll(), 5)
s.Options.Capabilities.MaximumMessageExpiryInterval = 0
s.Topics.Retained.Add("2/3/4", packets.Packet{Created: n - 8})
s.clearExpiredRetainedMessages(n)
require.Len(t, s.Topics.Retained.GetAll(), 6)
}
func TestServerClearExpiredClients(t *testing.T) {
@@ -3553,9 +3335,10 @@ func TestLoadServerInfoRestoreOnRestart(t *testing.T) {
require.Equal(t, int64(60), s.Info.BytesReceived)
}
func TestItoa(t *testing.T) {
func TestAtomicItoa(t *testing.T) {
i := int64(22)
require.Equal(t, "22", Int64toa(i))
ip := &i
require.Equal(t, "22", AtomicItoa(ip))
}
func TestServerSubscribe(t *testing.T) {