问题描述
我正在创建流分析应用程序，其中每个分析/功能都将作为微服务实现，以便以后可以在不同的项目中使用这种分析.
I am creating streaming analytics application in which each analytics/functionality will implement as a Microservice so that this analytics can able to use in the different project later.
我正在使用 Lagom 来创建微服务.我是拉戈姆的新手，这就是为什么我遇到了一些疑问.
I am using Lagom for creating Microservice. I am new in lagom that's why i came across with some doubts.
我不明白将我的数据流(来自多个传感器)发布到微服务，然后该微服务将数据发布到 kafka 主题的最佳方法是什么.
I don't understand what will be the best approach to Post my stream of data (coming from multiple sensors) to microservice and then this microservice publish data to kafka topic.
服务描述中是否有流消息的Lagom特性服务调用[源[字符串，未使用]，源[字符串，未使用]]这是从数百个 wifi 传感器发布数据流(大数据)的正确方法吗?它是否有能力处理这种近乎实时(= 5 秒)接收的大量数据流?
Does Lagom Feature of Stream Message in Service Description ServiceCall[ Source[String, NotUsed], Source[String, NotUsed]]is it the right approach to Post streams of data (Big Data) from hundreds of wifi sensors? Does it have a tenancy to deal with this huge amount of data streams receiving near real time (=5 Sec)?
其次，将数据发布到 kafka 主题时为什么我必须这样做实施持久性实体(Lagom 推荐)?因为卡夫卡本身保证至少一次传递消息
secondaly, while publish data to kafka topics why I have toimplement Persistent Entity (recommended by Lagom)? Because Kafkaitself guarantees at-least-once delivery of message
我的应用不是 CRUD 应用，它只支持处理流数据.
My application is not CRUD application, it only support to process streaming data.
推荐答案
Lagom 的流式调用使用 WebSockets.它建立在 Play 的 WebSocket 支持之上，可以扩展到数百万个连接的客户端.我不会将数百个 wifi 传感器称为大量数据，Lagom 应该可以轻松处理它，而且 Lagom 也可以轻松地进行水平扩展，因此如果您正在进行的处理很繁重，您可以轻松地将该处理分散到许多节点上.
Lagom's streaming calls use WebSockets. It's built on Play's WebSocket support, which can scale to millions of connected clients. I wouldn't call hundreds of wifi sensors a huge amount of data, Lagom should easily handle it, and Lagom can also easily be scaled horizontally, so if the processing you're doing is heavy, you can easily spread that processing across many nodes.
Lagom 目前不支持将传入的 WebSocket 流发布到 Kafka.虽然 Kafka 确实保证至少一次将消息发布到 Kafka，但在第一次将该消息发送到 Kafka 时没有这样的保证.例如，如果您执行副作用，例如更新数据库，然后发布消息，则无法保证如果应用程序在数据库更新和消息发布到 Kafka 之间崩溃，该消息最终会被发布到 Kafka(实际上不会，该消息会丢失).这就是为什么 Lagom 鼓励只将数据库事件流发布到 Kafka，因为以这种方式发布事件日志确实可以保证任何需要发送到 Kafka 的数据库操作至少发生一次.但是，如果您没有产生副作用(听起来好像没有)，那么这可能与您无关.在这种情况下，我建议直接使用 akka-streams-kafka(Lagom 的 Kafka 支持建立在它的基础上).
Publishing an incoming WebSocket stream to Kafka is not currently supported in Lagom. While Kafka does guarantee at least once once a message is published to Kafka, there are no such guarantees when getting that message into Kafka in the first instance. For example, if you do a side effect, such as update a database, then publish a message, there's no guarantee that if the application crashes between when the database is updated, and when the message is published to Kafka, that that message will eventually be published to Kafka (in fact it won't be, that message will be lost). This is why Lagom encourages only database event streams to be published to Kafka, since publishing the event log in that way does guarantee that any database operation that then needs to be sent to Kafka does happen at least once. However, if you're not doing side effects, which it sounds like you're not, then this might not be relevant to you. What I would recommend in that case would be to use akka-streams-kafka (what Lagom's Kafka support is built on) directly.
我在此处提出了一个参考您的用例的问题.
I've raised an issue referencing your use case here.
这篇关于在 Lagom 微服务中摄取流数据的最佳方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！