Commit Graph

114 Commits

Author SHA1 Message Date
Waldemar Quevedo
16d26f8e7b js: make js.Subscribe context aware
Can now attach a context to a subscription so that it is
unsubscribed and/or consumer deleted via propagation of
cancellation via parent context.

Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-12-14 13:13:23 -08:00
Bartosz Behring
67333f2fa1 Introducing ErrMsgAlreadyAckd to have more detailed error msg when trying to ack msg that was already acknowledged. 2021-11-14 10:28:36 +01:00
Casper Beyer
3d1af3f44c Fix typo in Bind documentation comments 2021-11-10 14:19:59 +08:00
R.I.Pienaar
2554593026 support tracing JS api calls
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-10-14 10:17:18 +02:00
Ivan Kozlovic
51d43e46a5 [FIXED] JetStream: ordered consumers handling of auto unsub
If user creates an ordered consumer and uses AutoUnsubscribe()
with the returned subscription, and if the ordered consumer was
reset (due to a gap detection), the library should resend an
UNSUB with adjusted max for the new subscription ID.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-10-13 13:59:40 -06:00
R.I.Pienaar
4b5b7cfbc1 add kv and object status functionality
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-10-12 16:54:44 +02:00
R.I.Pienaar
d7eb0d8f98 ensure get returns a nil and key not found as per spec
Signed-off-by: R.I.Pienaar <rip@devco.net>
2021-10-11 17:12:58 +02:00
Derek Collison
141643a344 [ADDED] KeyValue and ObjectStore support for JetStream.
Also:
- Fixed message reply in PublishMsgAsync
- Ability to seal streams
- Ability for consumer to get message headers only, no msg payload
- GetLastMsg and purgeStream by subject

Signed-off-by: Derek Collison <derek@nats.io>
2021-10-07 16:29:26 -06:00
Ivan Kozlovic
e30a74532e [FIXED] JetStream: queue name cannot contain "." when used as durable
When calling `js.QueueSubscribe[Sync](subject, queue_name)`, if
no durable name is provided, the queue name is used as the durable
name, so the same restriction applies

Resolves #840

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-10-07 11:56:25 -06:00
Ivan Kozlovic
630522759f Merge pull request #837 from nats-io/js_fix_fc
[FIXED] JetStream flow control may stall in some conditions
2021-10-05 16:31:09 -06:00
Ivan Kozlovic
c564a49724 Updates based on code review
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-10-05 15:14:47 -06:00
Waldemar Quevedo
8f86c1d37a js: Fix context usage with sub.Fetch and msg.Ack
The deadline of a context is now used to calculate
the time used for `expires` instead of the default `ttl`
of the JetStream context which was 5s.  This was preventing
library users from passing a context with a custom timeout.

This also disallows the usage of `context.Background`
to make it explicit that `sub.Fetch` has to be used
with a context that has a timeout since each fetch
request has to include an expire time anyway.

In case `context.WithCancel` is used, then a child context
with the same duration as the JetStream context default
timeout will be created.

Also in case msg.Ack it was possible to pass both timeout
and a context which would have been ambiguous and only
context option being used.

Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-10-05 13:28:14 -07:00
Ivan Kozlovic
66ba98e14f [FIXED] JetStream flow control may stall in some conditions
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-10-05 09:53:32 -06:00
Waldemar Quevedo
76690683f2 js: unblock batch requests on 408 with at least a message
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-09-16 14:33:58 -07:00
Ivan Kozlovic
8ec6f2c7b1 [UPDATED] JetStream go doc in the interface
Copied the go doc from implementation to interface, and reworded
some sections.

I decided to copy versus move because some IDE (for instance VS code)
seem to show the doc (when hovering) from the implementation, not
the interface.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-08 18:07:33 -06:00
Ivan Kozlovic
f3a531d051 [FIXED] Fetch() could return immediately with a timeout error
This would happen when pull requests would have filled the waiting
queue in the JetStream consumer and a 408 status was returned.

Resolves #809

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-09-02 08:12:36 -06:00
Ivan Kozlovic
33433c95d4 Fixed check for JS.ACK tokens count
Do not reject if it has only 11 tokens.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-25 09:57:01 -06:00
Ivan Kozlovic
d5b5768e78 [IMPROVED] Subscribe returns error if consumer config does not match
This is done to alert the user that consumer config is not changed
as one would expect.

This change is more tricky than expected. We can't simply compare
user's consumer config with what is returned from the server.
For once, there are configurations that may not be set by the
user that were set when the consumer was created, that the lib
should not fail. Let's say the consumer is created with a description,
why fail the user if they don't pass the Description() option at all?

The Description() option was actually missing, but this is a good
example. But same could be say with for instance "optional start
sequence". This may be something that is set when the consumer
is first created (possibly outside of the app with CLI), but when
user calls js.Subscribe(), that option should not have to be
set to the value that was used when creating the consumer.

I had to add "not set" values for replay and deliver policy, similar
to ack policy.

I added an extensive test that checks proper error when trying
to make configuration changes, but also that if the options are
not set, then the lib does not fail the subscribe call.

Resolves #796

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-23 17:11:31 -06:00
Ivan Kozlovic
d6590e7b27 [CHANGED] ConsumerInfo Delivered and AckFloor are now SequenceInfo
A new object `SequenceInfo` is added for `Delivered` and `AckFloor`'s
`ConsumerInfo` fields. This object contains `Last` which represents
the last activity time (in UTC).
Having this field under `SequencePair` was wrong since we used
`SequencePair` in other object, namely `MsgMetadata`, and in
that context, `Last` did not make sense.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-23 13:06:52 -06:00
Ivan Kozlovic
5a2962709c Removed new option SubjectIsDelivery and some other changes
This is to align with newer version of the ADR describing the
subscription workflow:
- Removed option SubjectIsDelivery that was introduced in main branch
(but not released yet).
- Take into consideration the new ACK layout
- Make sure that OrderedConsumer rejects user configuring DeliverSubject

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-18 17:48:05 -06:00
Ivan Kozlovic
9d3a000c8a Merge pull request #798 from nats-io/add_last_per_subject_delivery_policy
[ADDED] Last per subject delivery policy
2021-08-17 11:17:28 -06:00
Ivan Kozlovic
81f06013ba [ADDED] Last per subject delivery policy
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-17 09:56:42 -06:00
Ivan Kozlovic
ea700ff6ec [ADDED] ExpectLastSequencePerSubject() publish option
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-17 09:20:06 -06:00
Ivan Kozlovic
73d14e17c5 [ADDED] Description in StreamConfig and ConsumerConfig
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-16 19:20:42 -06:00
Ivan Kozlovic
4b8ebba8a3 Replaced BindDeliverSubject with SubjectIsDelivery
As discussed with Matthias who came up with the idea, this is
better because then we make use of the provided subject. Otherwise
it was looking weird to have something which meaning was:
```
js.SubscribeSync("ignored", nats.BindDeliverSubject("p.d4"))
```
Instead you would now have:
```
sub, err = js.SubscribeSync("p.d4", nats.SubjectIsDelivery())
```

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-16 18:54:34 -06:00
Ivan Kozlovic
e077154668 Addresses comments change requests based on code review
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-16 12:18:15 -06:00
Ivan Kozlovic
f730c0230b Add Domain to PubAck
Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-16 09:23:53 -06:00
Ivan Kozlovic
45b7e7725b JetStream: lot of changes
They will be described in the release notes, but gist:

Added:
- `DeliverSubject()` option to configure the deliver subject of a JetStream consumer created by the `js.Subscribe()` call (and variants)
- `BindDeliverSubject()` option to subscribe directly to a JetStream consumer deliver subject (bypassing any lookup or JetStream consumer creation)
- Fields `DeliverGroup` in `ConsumerConfig`, `PushBound` in `ConsumerInfo`. They help making prevent incorrect subscriptions to JetStream consumers
- Field `Last` in `SequencePair`

Changed:
- With a `PullSubscription`, calling `NextMsg()` or `NextMsgWithContext()` will now return `ErrTypeSubscription`. You must use the `Fetch()` API
- If the library created internally a JetStream consumer, the consumer will be deleted on `Unsubscribe()` or when the `Drain()` completes
- Fail multiple instances of a subscription on the same durable push consumer (only one active at a time). Also, consumers now have the concept of `DeliverGroup`, which is the queue group name they are created for. Only queue member from the same group can attach to this consumer, and a non queue subscription cannot attach to it. Note that this requires server v2.3.5
- Attempting to create a queue subscription with a consumer configuration that has idle heartbeats and/or flow control will now result in an error

Fixed:
- Possible lock inversion
- JetStream consumers could be incorrectly deleted on subscription's `Unsubscribe()`

Resolves #785
Resolves #776
Resolves #775
Resolves #748
Resolves #747

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-15 11:47:32 -06:00
Ivan Kozlovic
c91c1043a0 Fixed issue where JS would change subscription subject
This would cause issues during a reconnect

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-13 10:59:24 -06:00
Ivan Kozlovic
cfef5e6c50 Reworked PullSubscribe implementation
Use a SyncSubscription instead. The only visible change from the
user is that calling Fetch() after Unsubscribe() returns ErrBadSubscription
instead of timeout or context deadline exceeded, which makes more
sense to me. This is the only test that I had to tweak.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-08-09 17:59:00 -06:00
Derek Collison
74d305fc3b Updates based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-09 13:49:05 -07:00
Derek Collison
0b3386c62c Remove auto-unsub for now for fetch since it leaks subs in older servers
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-09 13:02:41 -07:00
Derek Collison
bce6f989f5 Changes to avoid lock inversion
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-09 12:53:14 -07:00
Derek Collison
c414086daa Move actual sub back to before consumer based on feedback
Signed-off-by: Derek Collison <derek@nats.io>
2021-08-09 12:48:18 -07:00
Derek Collison
90f007b828 Ordered Consumers
This introduces ordered consumers. They are a convenience over ephemeral, no ack, no redelivery, only deliver things in strict order setups.
We have the swap out when we detect gaps and change out the underlying sub and JetStream consumer, and we process the heartbeats as well, detecting gaps at the end or if a stream or consumer is pulled out from underneath of us.

Signed-off-by: Derek Collison <derek@nats.io>
2021-08-06 13:28:53 -07:00
Jaime Piña
d9b402d7d8 Update godoc 2021-07-14 10:39:34 -07:00
actatum
bb2c206532 Add two new sentinel errors. ErrNoStream and ErrNoConsumer (#760)
* squashed commit, adding sentinel errors ErrStreamNotFound and ErrConsumerNotFound

Co-authored-by: Waldemar Quevedo <waldemar.quevedo@gmail.com>
2021-06-28 12:02:06 -07:00
dtest11
c18fffce02 wrong note on ExpectLastMsgId && DeliverByStartSequencePolicy 2021-06-24 00:45:16 +08:00
dtest11
cd14d488ab fix Misspell on note AckSync func 2021-06-24 00:27:41 +08:00
Jarema
128e565e5c Add utility function for setting domain 2021-06-17 15:24:34 +02:00
Waldemar Quevedo
98dc666a0a js: Add ChanQueueSubscribe for JS
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-06-09 10:06:47 -07:00
Waldemar Quevedo
ba098b9960 Merge pull request #740 from nats-io/js-bind-consumer
js: Add nats.Bind option to disable creating consumers
2021-06-03 19:00:28 -07:00
Waldemar Quevedo
6237e10ee9 js: Add Bind option to Subscribe
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-06-03 17:38:19 -07:00
Matthias Hanel
efed691619 [changed] JetStream function to be a constructor only.
Removded then lookup of account info.
now already existing js.AccountInfo has to be called separately.

Signed-off-by: Matthias Hanel <mh@synadia.com>
2021-06-01 20:03:24 -04:00
Waldemar Quevedo
6b98b3edeb js: Improvements for push consumers with flow control
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-05-03 17:31:15 -07:00
Ivan Kozlovic
ea036d7532 Merge pull request #710 from wallyqs/godoc-context-example
Add godoc example for JS context functions
2021-05-03 12:42:53 -06:00
Ivan Kozlovic
1667213f01 Merge pull request #724 from nats-io/js_fix_shared_channel
Fixed SubscribeSync that could possibly share channel across 2 subs
2021-04-30 09:15:38 -06:00
Waldemar Quevedo
37f9684092 js: Add sub for PullSubscribe for fetch requests
Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-04-29 21:36:09 -07:00
Ivan Kozlovic
92272d3f21 Fixed SubscribeSync that could possibly share channel across 2 subs
Also added missing subscription's lock when setting some fields
at the end of initialization.

Fixed some flappers and missing defers.

Signed-off-by: Ivan Kozlovic <ivan@synadia.com>
2021-04-29 16:18:13 -06:00
Waldemar Quevedo
3f05b6aa47 Add nats.Header type based on http.Header
Currently it is case sensitive and preserves the case
without doing any normalization.

Signed-off-by: Waldemar Quevedo <wally@synadia.com>
2021-04-27 18:07:59 -07:00