开云体育

Hi Devs We are a startup and working on a software platform. We have decided to use wallaroo as a stream processor in our platform. We would like to try out the Go API instead of using python or pony API. However, I only found this https://github.com/WallarooLabs/blog.wallaroolabs.com/blob/master/content/post/golang-api.md So, are you discontinuing the go support? Else could you point me Go api documentation Thanks /Jay -- Jay Co-founder and CTO at xiges W: www.xiges.io E: dassanayake@... T: +94 115 92 6090 Delivery excellence guaranteed

Hi Just wanting to ask, what is the benefit of writing connector in pony rather than python ? So I manage writing connector for nats server (after digging through the source code), then I found that the connector is a separate process, if it is in pony, it will be in one process right ? The second, the connector api is marked experimental, any plan on breaking change ?

Hi everyone, I hope this message finds you well. I wanted to let everyone on the mailing list know that I'm now doing Developer Relations at Wallaroo Labs. What does this mean for you? Well, if you need help getting started with Wallaroo or making a Wallaroo project successful, come talk to me. I'd be thrilled to get emails from folks asking how Wallaroo can help solve their Python data processing problems. I'd love to dig into use cases with you. Whatever it is you need (within reason!), I'm here to help you with. Feel free to reach out either via the mailing list or to my email sean@.... -Sean-

Hi all, Today we released Wallaroo 0.6.1! The highlight of this release is the addition of stream windowing to the Wallaroo API! We do recommend upgrading and instructions can be found in our release notes. For additional details, please see our release notes: 0.6.1 Release Notes - Jonathan

Hi all, Today we released Wallaroo 0.6.0! The highlight of this release is a complete overhaul of the Wallaroo API to make it cleaner, simpler, and more intuitive. As a result of these changes, this is a breaking release. We do recommend upgrading and instructions can be found in our release notes. For additional details, please see our release notes: 0.6.0 Release Notes - Jonathan

Hello, This weeks blog post talks about using PostgreSQL with Wallaroo. It's a good starting point if you're interested in using Wallaroo with a SQL database. check it out here: https://blog.wallaroolabs.com/2018/11/using-wallaroo-with-postgresql/ As always contact us with any questions or feedback, Erik

Hi all, In the not so distant future, we are going to stop posting notices of new blog posts to the user mailing list. Why? We now have an option to subscribe to the blog. If you subscribe, you'll get notified when we publish new posts. You can sign up at https://blog.wallaroolabs.com/subscribe/ -Sean-

Happy Friday, everyone. I'm the author of a blog article published today, "The Treacherous Tangle of Redundant Data: resilience for Wallaroo". A few weeks ago, John Mumm wrote an article that described what Wallaroo does to be resilient in case of a rebootable crash. But what if the crashed worker cannot reboot? Today's article describes the data redundancy technique that permits a Wallaroo cluster to recover after a crash with catastrophic data loss. Today's article: https://blog.wallaroolabs.com/2018/11/the-treacherous-tangle-of-redundant-data-resilience-for-wallaroo/ John's article: https://blog.wallaroolabs.com/2018/10/checkpointing-and-consistent-recovery-lines-how-we-handle-failure-in-wallaroo/ -Scott

Hi all, Today we released Wallaroo 0.5.4. The highlight of 0.5.4 is support for Python 3. Users can now use `machida3` to develop Wallaroo applications written in Python 3, see our latest documentation to get started. Although this is a preview release, we are very excited to get it into your hands. This is a patch release, meaning there are no breaking changes to the existing API. This allows you to drop in your existing Python 2 application(s) into the latest release and take full advantage of bug fixes we’ve made since our last release. Full details available in the release notes: 0.5.4 Release Notes - Jonathan

Hello, this weeks blog post gives an overview of our new Connector APIs. These APIs make connecting to external applications much easier than before. We're excited to hear your feedback https://blog.wallaroolabs.com/2018/10/introducing-connectors-wallaroos-window-to-the-world/

https://blog.wallaroolabs.com/2018/10/spinning-up-a-wallaroo-cluster-is-easy/ If you're looking into setting up a Wallaroo cluster, take a look at how easy it is to do with Pulumi + Ansible. Cheers, Simon Zelazny

Hey everyone, We just put up a new blog post that discusses our recent asynchronous checkpointing work released in 0.5.3. It goes into some detail about the problems around recovering distributed systems to a consistent global state, some ways you might do this incorrectly, and some reasons why the checkpointing algorithm we settled on is a great fit for streaming systems. Check it out: https://blog.wallaroolabs.com/2018/10/checkpointing-and-consistent-recovery-lines-how-we-handle-failure-in-wallaroo/

Previously most of the code base was Apache 2.0 licensed but some was "source available" and under a non-open source license. Full details in my blog post from last week: https://blog.wallaroolabs.com/2018/10/wallaroo-goes-full-apache-2.0/

Hi all, Today we released Wallaroo 0.5.3. This is a patch release that includes two very important new features. First, we've released a preview version of the Python Connector API. This allows developers to build sources and sinks without the need to worry about Wallaroo’s internal protocol. We also have a better resilience story: we now use an algorithm based on the Chandy-Lamport snapshotting algorithm that minimizes the impact of checkpointing on processing in-flight messages. Full details available in the release notes: 0.5.3 Release Notes -Dipin

Hi everyone, It's Thursday, and that's (usually) blog day! This week's post is about paralellizing pandas batch jobs with Wallaroo: https://blog.wallaroolabs.com/2018/09/make-python-pandas-go-fast/ Cheers, Simon Zelazny

Hi all, We have a new blog post about "Streamlining the Wallaroo installation process with Wallaroo Up." You can find it at: https://blog.wallaroolabs.com/2018/08/wallaroo-up-automating-the-linux-developer-experience-for-wallaroo/ -Dipin

Hi, everyone. The folks at InfoQ have published an article that I've written about the new consistent hashing technique that is being added to Wallaroo. The 0.5.0 Wallaroo release added support for "dynamic keys" [1]. Wallaroo can now automatically recognize new keys and route data to the appropriate partitioned state step. The routing algorithm used today (including the recent 0.5.2 release) is very general and is unaware of load balancing considerations. We are now integrating a technique called Random Slicing into Wallaroo to add very fine-grained control over the distribution of keys across Wallaroo worker processes. The second half of the InfoQ article [2] describes the Random Slicing consistent hashing technique, together with several illustrations to show it adapts to changing cluster membership and load balancing criteria. The first half of the article describes some earlier consistent hashing techniques, including the one used by Amazon's original Dynamo database and adopted by Riak and other distributed databases. John Mumm, Andy Turley, and I are excited to bring enhancements to Wallaroo's dynamic keys, load balancing, and crash resilience features to Wallaroo in the next month. If you have questions about any of this work, please feel free to contact me by email. -Scott [1] https://github.com/WallarooLabs/wallaroo/releases/tag/0.5.0 [2] https://www.infoq.com/articles/dynamo-riak-random-slicing

Hello everyone, We have a new blog post on another stream processing use case, analyzing trends. You can read it here: https://blog.wallaroolabs.com/2018/08/real-time-streaming-pattern-analyzing-trends/ Otherwise enjoy your weekend!

Hi all, Today we released Wallaroo 0.5.2. We've simplified the Wallaroo installation process on Linux and added support for additional Linux distributions via Wallaroo Up, a shell script that automates the from-source install of Wallaroo. We've also added support for Wallaroo Go in Docker and Vagrant. Full details available in the release notes: 0.5.2 Release Notes - Jonathan

Hi all, This week, I wrote about how we use Elixir as a lightweight tool to store Wallaroo's real-time metrics data. Enjoy! https://blog.wallaroolabs.com/2018/08/utilizing-elixir-as-a-lightweight-tool-to-store-real-time-metrics-data/