New blog post: Some Common Mitigation Techniques for Overload in Queueing Networks
Hi, everyone.? The latest in the Wallaroo Labs blog series was published this morning. It's an article I wrote that presents a brief overview of queueing networks and what the term overload means for a queueing network, followed by an outline of some common techniques that computer systems use to mitigate the effects of overload.
It's the first part of a two part series.? The second article will be published next week, on Tuesday, April 3, 2018.? That article looks very specifically at Wallaroo and how it uses back-pressure to handle overload, i.e., data sinks that are slower than Wallaroo, or data sources that are faster than Wallaroo.
Thanks for reading!
-Scott
|
Hi all,
Today we released Wallaroo 0.4.1.
Full details available in the release notes:?
- Jonathan
|
How we performance test Wallaroo
Hi folks,
Today we've published a blog post about how we performance test Wallaroo, our high-performance, low-latency stream processing system. I talk about the how we isolate Wallaroo from system interference and the application we use for our performance testing.
It can be found at:?
-Dipin
|
How We Built Wallaroo to Process Millions of Messages/Sec with Microsecond Latencies
Hi everyone,
I wrote a post about how we approached designing Wallaroo with performance in mind and some principles that could be useful when building your own performance-sensitive software systems:?
- John
|
Building low-overhead metrics collection for high-performance systems
Hi all,
I wrote a post about the high-level design decisions behind the metrics capturing system used by Wallaroo to maintain a low-overhead and how some of those decisions can apply to other high-performance?systems:?
If you care to get more of a lower-level breakdown of the design choices, Nisan wrote a great post on that:?https://blog.wallaroolabs.com/2018/02/latency-histograms-and-percentile-distributions-in-wallaroo-performance-metrics/
- Jonathan
|
A Scikit-learn pipeline in Wallaroo
Hi all,
We have a new blog post today. Alan Mosca wrote it. I'm announcing it because he is off in the Python mines cooking up some performance improvements.
Today's post show's how you can use Scikit-learn trained models inside a Wallaroo application. Cool stuff:
https//blog.wallaroolabs.com/2018/02/a-scikit-learn-pipeline-in-wallaroo/
-Sean0
|
Idiomatic Python Stream Processing in Wallaroo
Hey all,
I wrote a blog post about our effort to make the Wallaroo Python API more Pythonic.
Please take a look and let us know if you have any feedback on the post, the API, or anything else.
andy
|
New blog post about "Why we wrote our Kafka Client in Pony"
Hi folks,
Today we've published a blog post about why we wrote our Kafka Client in Pony instead of relying on the existing C Kafka Client and creating bindings for it. We talk about the pros/cons of our decision and how far we¡¯ve come including some preliminary performance numbers.
It can be found at:
|
blog post for our new Go API
Hello Wallaroo users,
You may have noticed that we recently put out a Go API for Wallaroo. Today we've published a blog post that describes the API and shows how to use it.
Please have a look and let us know if you have any feedback. This is an initial release, so we're excited to hear what you think.
Andrew Turley
|
New Blog post: "You might have a streaming data problem if..."
Over the last few years, I've had many conversations with folks about when they need a streaming data tool and when they need a batch processing tool. One conclusion I've come to is that we draw a false dichotomy between the two. In today's blog post, I've written a bit about that. Hopefully you find it interesting and enlightening.
-Sean-
|
Wallaroo 0.4.0 has been released!
Hi y'all,
Wallaroo 0.4.0 has been released. For the impatient, you can dive right into the release notes at:
Major highlights from this release include:
- New more Pythonic API.
- The first public release our autoscaling feature aka dynamic cluster resizing
The new Python API is a breaking change. You'll need to update applications to use it. It should be pretty straightforward to update. Alan Mosca has put together a blog post to guide you through the process:
Enjoy!
|
New Blog Post: "Serverless, Scale-Independent Processing, and Wallaroo"
Morning!
Well, it's morning for me here in New York City. Happy time of day to you whereever you happen to be. We have a new blog post for you!
When we first start talking to folks about what Wallaroo is and does, some of them gravitate towards the term "serverless" and ask us how Wallaroo is different. At first, we would respond that we "aren't really like serverless because...". Over time, we realized that we were saying that because we were thinking about serverless through the lens of current implementations. We've come to believe that there are some goals that serverless embodies that transcend the limitations of current implementation. Worthy goals. Things we should aspire to. Today blog post "Serverless, Scale-Independent Processing, and Wallaroo" is our first step towards trying to create a more expansive concept of what serverless could mean. Hope you enjoy.
|
Wallaroo 0.3.3 has been released!
Hi all,
We just released version 0.3.3 of Wallaroo. The big addition in this release is a preview version of our new Go API. We are quite excited to get people using it. We're looking forward to your feedback and growing the API with you.
Release notes are available at
|
Wallaroo 0.3.2 has been released
Today we released Wallaroo 0.3.2. Includes several updates to our documentation.
Full details available in the release notes:?
- Jonathan
|
Wallaroo 0.3.1 has been released
Today we released Wallaroo 0.3.1. Includes several updates to our documentation.
Full details available in the release notes:?
https://github.com/WallarooLabs/wallaroo/releases/tag/0.3.1
- Jonathan
|
New blog post: "Simplify Stream Processing in Python and Wallaroo using Docker"
https://blog.wallaroolabs.com/2017/12/simplify-stream-processing-in-python-and-wallaroo-using-docker/
|
Wallaroo 0.3.0 has been released
Today we released Wallaroo 0.3.0. A major addition includes support for getting started with Wallaroo in Docker.
Full details available in the release notes:
https://github.com/WallarooLabs/wallaroo/releases/tag/0.3.0
- Jonathan
|
New Blog Post: Dynamic tracing a Pony + Python program with DTrace
Good morning.? This week's new blog introduces a topic close to my heart, dynamic tracing.? The link is:
|
New Blog Post: "Stateful Multi-Stream Processing in Python with Wallaroo"
|
New blog post: Non-native Event-driven windowing in Wallaroo
Hi all,
I just authored a new blog post that explains how to create windows over data streams using wallaroo, without the need for any native support. You can find the blog here: Please let me know your thoughts, comments and questions either here or on IRC (#wallaroo on Freenode)!
Thanks, Alan
|
