Date: Fri, 20 Jan 2023 10:12:26 -0600
Agree with Timur and Wesley. I work in HFT, MoodyCamel seems to be the go
to lock-free mpmc queue at most places. There are some good hand-rolled
implementations out there too, example:
https://github.com/erez-strauss/lockfree_mpmc_queue
On Fri, Jan 20, 2023 at 10:06 AM Wesley Maness via SG14 <
sg14_at_[hidden]> wrote:
> +1 to Timur.
>
> I can speak to MoodyCamel as well, it found some great usage in core
> trading frameworks (event based) of a few shops that I know of personally.
>
> On Fri, Jan 20, 2023 at 10:55 AM Timur Doumler via SG14 <
> sg14_at_[hidden]> wrote:
>
>> Hi Honey,
>>
>> In audio processing, lock-free queues are among the most frequently used
>> data structures. Often, we use single-producer single-consumer queues as
>> they have the desirable property to be wait-free, not just lock-free (a
>> requirement for hard real time programming). But use cases for
>> multi-producer multi-consumer queues exist also in this industry. Some
>> companies roll their own; the two off-the-shelf solutions that I have seen
>> being used in production are Boost and Moodycamel (
>> https://github.com/cameron314/concurrentqueue). I found the Moodycamel
>> one to have better performance than the Boost one.
>>
>> Hope this is helpful.
>>
>> Cheers,
>> Timur
>>
>> On 18 Jan 2023, at 15:49, Honey Sukesan via SG14 <sg14_at_[hidden]>
>> wrote:
>>
>>
>> Hi Scott,
>>
>> Thank you for your reply.
>> Yes
>> , it's a multithreaded program. I believe multiprocess constructs could
>> be used along side threads as well. I've tried benchmarking using
>> std::queue protected with a std:: unique_lock & condition_var for the same
>> scenario. But performance results was not that great. Let me know what you
>> think.
>>
>> Best,
>> Honey
>>
>>
>> On Wed 18 Jan 2023, 8:01 a.m. Scott Michaud, <scott_at_[hidden]> wrote:
>>
>>> You say that you have a producer-consumer scenario with multiple
>>> *threads*. Are they all in the same executable / process?
>>>
>>> On Tue, Jan 17, 2023 at 10:52 PM Honey Sukesan via SG14 <
>>> sg14_at_[hidden]> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Hope all is well.
>>>>
>>>> This is my first email to the list. Not sure if it's the place to ask
>>>> general technical questions. Please accept my apologies if I'm violating
>>>> any rules of this study group.
>>>>
>>>> I would like to seek an opinion regarding boost:: interprocess::
>>>> message_queue vs boost::lockfree::queue.
>>>>
>>>> Has anyone used boost::lockfree::queue or any of Boost.LockFree data
>>>> structures in any embedded platforms?
>>>>
>>>> I have a producer-consumer scenario involving multiple threads. Tried
>>>> out a sample code using int values pushing to/consuming from the queue &
>>>> analysed elapsed time between push/pop on a Linux x86. It's found
>>>> boost::lockfree::queue gives better time performance results compared to
>>>> boost:: interprocess:: message_queue.
>>>>
>>>> Haven't tried this on actual hardware target yet. Our hardware target
>>>> is an ARM based processor running QNX RTOS.
>>>>
>>>> I'm listening to Herb's CppCon talk on Lock-free programming now.
>>>>
>>>> It would be great if any of you has any other recommendations/pros or
>>>> cons I need to be aware of before I fix boost::lockfree::queue rather than
>>>> boost:: interprocess:: message_queue in our production code?
>>>>
>>>> Thank you in advance,
>>>> Best,
>>>> Honey
>>>>
>>>> Honey Sukesan
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> SG14 mailing list
>>>> SG14_at_[hidden]
>>>> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>>>>
>>> _______________________________________________
>> SG14 mailing list
>> SG14_at_[hidden]
>> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>>
>> _______________________________________________
>> SG14 mailing list
>> SG14_at_[hidden]
>> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>>
>
>
> --
> Wesley C. Maness
> 347-387-7162
> _______________________________________________
> SG14 mailing list
> SG14_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>
to lock-free mpmc queue at most places. There are some good hand-rolled
implementations out there too, example:
https://github.com/erez-strauss/lockfree_mpmc_queue
On Fri, Jan 20, 2023 at 10:06 AM Wesley Maness via SG14 <
sg14_at_[hidden]> wrote:
> +1 to Timur.
>
> I can speak to MoodyCamel as well, it found some great usage in core
> trading frameworks (event based) of a few shops that I know of personally.
>
> On Fri, Jan 20, 2023 at 10:55 AM Timur Doumler via SG14 <
> sg14_at_[hidden]> wrote:
>
>> Hi Honey,
>>
>> In audio processing, lock-free queues are among the most frequently used
>> data structures. Often, we use single-producer single-consumer queues as
>> they have the desirable property to be wait-free, not just lock-free (a
>> requirement for hard real time programming). But use cases for
>> multi-producer multi-consumer queues exist also in this industry. Some
>> companies roll their own; the two off-the-shelf solutions that I have seen
>> being used in production are Boost and Moodycamel (
>> https://github.com/cameron314/concurrentqueue). I found the Moodycamel
>> one to have better performance than the Boost one.
>>
>> Hope this is helpful.
>>
>> Cheers,
>> Timur
>>
>> On 18 Jan 2023, at 15:49, Honey Sukesan via SG14 <sg14_at_[hidden]>
>> wrote:
>>
>>
>> Hi Scott,
>>
>> Thank you for your reply.
>> Yes
>> , it's a multithreaded program. I believe multiprocess constructs could
>> be used along side threads as well. I've tried benchmarking using
>> std::queue protected with a std:: unique_lock & condition_var for the same
>> scenario. But performance results was not that great. Let me know what you
>> think.
>>
>> Best,
>> Honey
>>
>>
>> On Wed 18 Jan 2023, 8:01 a.m. Scott Michaud, <scott_at_[hidden]> wrote:
>>
>>> You say that you have a producer-consumer scenario with multiple
>>> *threads*. Are they all in the same executable / process?
>>>
>>> On Tue, Jan 17, 2023 at 10:52 PM Honey Sukesan via SG14 <
>>> sg14_at_[hidden]> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Hope all is well.
>>>>
>>>> This is my first email to the list. Not sure if it's the place to ask
>>>> general technical questions. Please accept my apologies if I'm violating
>>>> any rules of this study group.
>>>>
>>>> I would like to seek an opinion regarding boost:: interprocess::
>>>> message_queue vs boost::lockfree::queue.
>>>>
>>>> Has anyone used boost::lockfree::queue or any of Boost.LockFree data
>>>> structures in any embedded platforms?
>>>>
>>>> I have a producer-consumer scenario involving multiple threads. Tried
>>>> out a sample code using int values pushing to/consuming from the queue &
>>>> analysed elapsed time between push/pop on a Linux x86. It's found
>>>> boost::lockfree::queue gives better time performance results compared to
>>>> boost:: interprocess:: message_queue.
>>>>
>>>> Haven't tried this on actual hardware target yet. Our hardware target
>>>> is an ARM based processor running QNX RTOS.
>>>>
>>>> I'm listening to Herb's CppCon talk on Lock-free programming now.
>>>>
>>>> It would be great if any of you has any other recommendations/pros or
>>>> cons I need to be aware of before I fix boost::lockfree::queue rather than
>>>> boost:: interprocess:: message_queue in our production code?
>>>>
>>>> Thank you in advance,
>>>> Best,
>>>> Honey
>>>>
>>>> Honey Sukesan
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> SG14 mailing list
>>>> SG14_at_[hidden]
>>>> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>>>>
>>> _______________________________________________
>> SG14 mailing list
>> SG14_at_[hidden]
>> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>>
>> _______________________________________________
>> SG14 mailing list
>> SG14_at_[hidden]
>> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>>
>
>
> --
> Wesley C. Maness
> 347-387-7162
> _______________________________________________
> SG14 mailing list
> SG14_at_[hidden]
> https://lists.isocpp.org/mailman/listinfo.cgi/sg14
>
Received on 2023-01-20 16:12:38