标签归档:queue

golang queue packages

github.com/rylio/queue

Execute tasks in parallel with a concurrency limit

github.com/bgentry/que-go

An interoperable Golang port of the Ruby Que queuing library for PostgreSQL

goworker

goworker is a Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers.

goworker is compatible with Resque, so you can push your jobs with Rails and Resque, and consume them with Go in the background.

machinery

Machinery is an asynchronous task queue/job queue based on distributed message passing.

golang通过web管理worker

Writing worker queues, in Go

Have you ever wanted to write something that is highly concurrent, and performs as many tasks as you will let it, in parallel? Well, look no further, here is a guide on how to do just that, in Go!

go-workers - Sidekiq compatible background workers in golang

Sidekiq compatible background workers in golang.

  • reliable queueing for all queues using brpoplpush
  • handles retries
  • support custom middleware
  • customize concurrency per queue
  • responds to Unix signals to safely wait for jobs to finish before exiting.
  • provides stats on what jobs are currently running
  • well tested

NSQ - A realtime distributed messaging platform

NSQ is a successor to simplequeue (part of simplehttp) and as such is designed to (in no particular order):

  • support topologies that enable high-availability and eliminate SPOFs
  • address the need for stronger message delivery guarantees
  • bound the memory footprint of a single process (by persisting some messages to disk)
  • greatly simplify configuration requirements for producers and consumers
  • provide a straightforward upgrade path
  • improve efficiency

Background Jobs with Que-Go

Web servers should focus on serving users as quickly as possible. Any non-trivial work that could slow down your user’s experience should be done asynchronously outside of the web process.

Worker queues are commonly used to accomplish this goal. For a more in-depth description of the worker queue architectural pattern, read the Worker Dynos, Background Jobs and Queueing article. Here we’ll demonstrate this pattern using a sample Go application using Que-Go and Heroku Postgres.