Within Streams Publish/Subscribe is simplification/enhancement of Import/Export that is designed to work across all programming models.
Thus (for example) an SPL application can publish a stream and a Python or Java application can subscribe to that stream to process the tuples.
A published stream has a topic, e.g. streamsx/transportation/nextbus/sfmuni/locations and a type which is one of:
- Structured (SPL) schema, e.g. tuple<rstring busid, float64 latitude, float64 longitude, float64 speed>
- String - each tuple is a string
- JSON - each tuple is a JSON object
- Binary- each tuple is binary data
- XML - each tuple is an XML document
- Java object (Java/Scala apps only)
- Python object (Python apps only)
An application subscribes to published streams using a topic filter and a type, so it could be:
streamsx/transportation/nextbus/sfmuni/locations to select the above topic, or:
streamsx/transportation/nextbus/+/locations to select any published stream from different bus agencies, e.g. matching:
(topics are based upon the MQTT topic scheme).
Adding Publish/Subscribe to Streams Designer would provide seamless integration with existing SPL applications and allow Streams Designer to fit in the role of ad-hoc analytics integrating with "mission-critical" applications, the USG pattern described by Brian Williams in Austin.
From an implementation point of view:
- Publish/Subscribe is simply invoking SPL operators com.ibm.streamsx.topology.topic::Publish/Subscribe
- Available published streams can be determined from the Streams REST api - we have Python code that does this (for use in DSX), so a user can see the available streams.
Thus I imagine that the editor for subscribe could show a list of available topics (similar to MessageHub), maybe presenting them as "live feeds" ? The type of the schema is predefined from the published stream. It could also suggest wildcard topics, based upon available streams.
For publish the editor would allow the topic to be entered and the published type, e.g. the current schema, or convert to JSON.