Category Archives: amqp

WSGI+DJANGO+Celery cannot import spec

I have set a right configurations for celery according documentation. And tested by using "run server", there is no problem.

wsgi.py:

import site
site.addsitedir('/usr/local/lib/python2.7/site-packages/')

import os
import sys
sys.path.append('opt/lampp/htdocs/dpcm/dpcm1')
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "dpcm1.settings")
os.environ.setdefault("PYTHON_EGG_CACHE", "/home/inf/Documents/.python-eggs")

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

However once start apache, it will generate an importerror:

Traceback (most recent call last):
File "/opt/lampp/htdocs/dpcm/dpcm1/wsgi.py", line 21, in <module>
    application = get_wsgi_application()
File "/usr/local/lib/python2.7/site-packages/django/core/wsgi.py", line 13,in get_wsgi_application
    django.setup(set_prefix=False)
File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 22,in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
    self._setup(name)
File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 41, in _setup
    self._wrapped = Settings(settings_module)
File "/usr/local/lib/python2.7/site-packages/django/conf/__init__.py", line 97, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
File "/opt/lampp/htdocs/dpcm/dpcm1/__init__.py", line 5, in <module>
    from .cey import app as celery_app
File "/opt/lampp/htdocs/dpcm/dpcm1/cey.py", line 3, in <module>
    from celery import Celery
File "/usr/local/lib/python2.7/site-packages/celery/local.py", line 509, in __getattr__
    module = __import__(self._object_origins[name], None, None, [name])
File "/usr/local/lib/python2.7/site-packages/celery/app/__init__.py", line 5, in <module>
    from celery import _state
File "/usr/local/lib/python2.7/site-packages/celery/_state.py", line 15, in <module>
    from celery.utils.threads import LocalStack
File "/usr/local/lib/python2.7/site-packages/celery/utils/__init__.py", line 10, in <module>
    from .nodenames import worker_direct, nodename, nodesplit
File "/usr/local/lib/python2.7/site-packages/celery/utils/nodenames.py", line 7, in <module>
    from kombu.entity import Exchange, Queue
File "/usr/local/lib/python2.7/site-packages/kombu/entity.py", line 6, in <module>
    from .abstract import MaybeChannelBound, Object
File "/usr/local/lib/python2.7/site-packages/kombu/abstract.py", line 6, in <module>
    from .connection import maybe_channel
File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 15, in <module>
    from kombu import exceptions
File "/usr/local/lib/python2.7/site-packages/kombu/exceptions.py", line 6, in <module>
    from amqp import ChannelError, ConnectionError, ResourceError
File "/usr/local/lib/python2.7/site-packages/amqp/__init__.py", line 47, in <module>
    from .connection import Connection  # noqa
File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 29, in <module>
    from . import spec
ImportError: cannot import name spec

Is there a way to bind a QPID header exchange and a queue when a specific header parameter does not exist?

Lets say I have a QPID exhange called: exc.output and a queue named general.input. I want to forward messages from exchange to queue only if a specific header parameter (eg: type) is either missing or is an empty string.

Example message:

body: test message
contentType: text/plain
headers: {
    timestamp: 2016-03-23T14:00:00
}

or:

body: test message
contentType: text/plain
headers: {
    timestamp: 2016-03-23T14:00:00,
    type: ''
}

But this should not be forwarded because the type header parameter exists and has a meaningful value:

body: test message
contentType: text/plain
headers: {
    timestamp: 2016-03-23T14:00:00,
    type: 'original'
}

I can handle the empty string situation with this command: qpid-config bind exc.output general.input general.input.binding all type=

I didn't find a way to check for missing parameters. Something like this would be great: qpid-config bind exc.output general.input general.input.binding any type=, !type

Here is the manual for qpid: https://qpid.apache.org/releases/qpid-cpp-0.34/cpp-broker/book/chapter-Managing-CPP-Broker.html#MgmtC-2B-2B-Usingqpidconfig

Accessing RabbitMQ through webserver

I have a RabbitMQ server running on an EC2 instance, and a URL address pointing to the instance configured in Route53, which I use to access the RabbitMQ web UI. I can access the UI just fine using Nginx or Apache, but when trying to look at the page of a specific queue or exchange I receive an error saying the object was not found. This behaviour does not happen when I access the UI directly through IP:Port url.

I've noticed this is a known bug, which was addressed here - https://fatalfailure.wordpress.com/2012/11/16/nginx-as-reverse-proxy-for-rabbitmq-mochiweb-server/

I've tried implementing the fix listed in this post, using the following Apache configuration -

<VirtualHost *:80>
    ProxyPreserveHost On
    ProxyRequests Off
    ServerName stage-rabbithq.apester.com 
    ServerAlias stage-rabbithq.apester.com
    ProxyPass /api http://localhost:15672/api nocanon
    ProxyPass / http://localhost:15672/
    ProxyPassReverse / http://localhost:15672/
</VirtualHost>

But the error prevails. Does anybody have a solution for this error?

Camel AMQP transactions issue (with Qpid client)

I am trying to add transactions over an AMQP connection which I created for connecting to an external AMQP broker (which I do not have access to) from my project.

In order to connect to the AMQP broker with SSL I am using a ConnectionFactory implementation from Qpid: org.apache.qpid.amqp_1_0.jms.impl.ConnectionFactoryImpl.

I am connecting with something like below:

  • URI: amqps://x.x.x.x:port?brokerlist='ssl://x.x.x.x:port'

  • topic: topic://topicName

  • a truststore and a keystore

The connection is initialized from an Apache Camel (v2.14.1) route, with the Camel AMQP component which was built on top of JMS, exactly as stated in the AMQP Component documentation from Apache Camel (which can be found here: http://camel.apache.org/amqp.html in section "Using Topics").

The AMQP component works perfectly fine when I use it without transactions enabled.

If, on the other hand, I enable transactions by:

  • providing a transaction manager which has to be an implementation of the interface org.springframework.transaction.PlatformTransactionManager as org.springframework.transaction.jta.JtaTransactionManager

  • setting transacted=true configuration option on the Camel endpoint

  • setting cacheLevelName=CACHE_NONE on the Camel endpoint somehow only one messaged gets picked up from the topic, everything else is lost/ignored.

The logs show that the transaction manager is created successfully:

DEBUG org.apache.camel.util.IntrospectionSupport: Configured property: transactionManager on bean: [email protected] with value: org.springframework.transaction.jta.JtaTransactionMana[email protected]

DEBUG org.apache.camel.util.IntrospectionSupport: Configured property: cacheLevelName on bean: [email protected] with value: CACHE_NONE

DEBUG org.apache.camel.util.IntrospectionSupport: Configured property: concurrentConsumers on bean: [email protected] with value: 1

DEBUG org.apache.camel.util.IntrospectionSupport: Configured property: transacted on bean: [email protected] with value: true

Would you please let me know what am I doing wrong?

(I suspect that maybe the broker I am connecting to does not support transactions)

Class ‘AMQPConnection’ not found; AMQP is installed, but not shown in phpinfo

I am receiving the error:

Fatal error: Class 'AMQPConnection' not found

I have installed it following http://php.net/manual/fa/amqp.installation.php

and:

extension=amqp.so

has been added to the main php.ini file and also the virtualhost's own.

and apache has been restarted.

On phpinfo, amqp appears under "Additional ini files parsed":

/etc/php5/cgi/conf.d/05-opcache.ini, /etc/php5/cgi/conf.d/10-pdo.ini, /etc/php5/cgi/conf.d/20-curl.ini, /etc/php5/cgi/conf.d/20-gd.ini, /etc/php5/cgi/conf.d/20-imap.ini, /etc/php5/cgi/conf.d/20-json.ini, /etc/php5/cgi/conf.d/20-mysql.ini, /etc/php5/cgi/conf.d/20-mysqli.ini, /etc/php5/cgi/conf.d/20-pdo_mysql.ini, /etc/php5/cgi/conf.d/20-readline.ini, /etc/php5/cgi/conf.d/25-amqp.ini

However the main amqp block which should appear under Configuration doesn't exist.

So it's almost like it's installed but not started / installed correctly.

When restarting apache it gives no errors. So I am not sure where to look for any errors / reasons why it might not be starting or loading into phpinfo.

Any pointers, most appreciated.

Thanks

Conversion from MQTT to AMQP/STOMP

I'm kind of new to these protocols, and just started exploring Message brokers like Apache Apollo and RabbitMQ. So my broker receives MQTT messages from a publisher. And I would like to convert it into AMQP (preferably) or STOMP protocol to send to a web server. But I've so far been unable to do so. I looked into RabbitMQ, and tried enabling the MQTT plugin, but when I do load it, I'm unable to start the server. I was wondering if anyone can guide me here? Is there an API that can help me? And I'm very confused about RabbitMQ. I've been able to load other plugins easily,like stomp, management utilities etc.