ESB and Content-Encoding = "gzip"

Posted by Maciej.Barcz@ottogroup.com on 13-Jul-2010 04:31

hello,

i'm trying to implement a custom service talking to my backend systems (a kind of soap callout service). One of the requirements is to work with compressed payload.

Our clients support gzip (Accept-Encoding = "gzip"), same for our backend system (Content-Encoding = "gzip").  Without ESB everything is pretty easy - as long as the http headers are correctly set, everything works as designed and we can work with compressed payloads.

Now, we want to place in between a piece of Sonic ESB (7.6) - in such a case our clients will talk to the HTTP Acceptor (request won't be compressed, however the response should be, thus the client sets "Accept-Encoding = gzip"), the HTTP Requests will be routed as XQMessages to the custom service and send to the backend system as uncompressed HTTP Requests. Backend will respond with gziped payload, custom service will read the payload as a byte[], put the byte[] to the XQMessage and place the message in the outbox.

Problem 1:  ESB for some reason doesn't propagate the "Accept-Encoding" header - my custom service does not know, if the compressed responses will be unsterstood. I can workaround this problem by setting my own header (custom headers, set by the client, will be propagated "down" to the service).

Problem 2: However, even if I find out the way to inform my custom service if the client does accept the compressed responses, I don't how, how the returing message should look like, to carry the gziped-response (Content-Encoding = gzip, Content-Type = text/xml, Single-Part) back to the client. Setting the Message Type to text/xml I'm getting grabbled payload (cannot be unziped anymore - it looks for me, that the HTTP Acceptor is converting my bytes back to String, which is almost always a bad idea).

Do you know, how to solve such a problem?

Do you know, if there is a way to configure/instrument the embedded jetty to enable support for gziped payloads (for example with a help of well-known gzip-filters)?

I would be very grateful for any hints.

regards,
maciej

All Replies

Posted by jerem on 13-Jul-2010 10:34

Marciej,

Problem 1: How do you forward the message to the outbox? If you create a new message

context.getMessageFactory().createMessage()

you have to retrieve the properties in the incoming message and put it in the outbound message.

You can also resend the incoming message with a replaced part.

Problem 2 : Set content-type to byte

note that a byte payload won't be human readable.

Try to look at the compression service code and other custom services from codeshare. This will certainly help you.

Regards

Chan Jeremy

Posted by Maciej.Barcz@ottogroup.com on 21-Jul-2010 03:29

Hi,

content doesn't have to be human readable. Setting content-type to byte is not enough.
I also don't have any problems implementing custom service with gzip support.

Let me explain my scenario:

[client] [esb http acceptor] XQMessage [esb custom service] [backend]

Requirements:

1. (client->esb) requests doesn't have to be compressed, however the Accept-Encoding header should be preserved

2. (esb->backend) request doesn't have to be compressed, however the Accept-Encoding header should be preserved

3. (esb

4. (client

The most imporant is, that the response-message on his way back to the caller won't be destroyed by HTTP Acceptor.

Progress ESB doesn't support "Accept-Encoding" headers - such a header (a.o.) will be silently removed by the HTTP Acceptor from the incoming message.

Now, having a custom service that is able to talk "gzip" to my backend, I have to make sure, that my message (Content-Type=text/xml, Content-Encoding=gzip) will be routed back to the called thru the HTTP Acceptor (content-reply) and *never ever won't be changed*. Otherwise my client won't be able to unzip the payload. My understanding is, that the HTTP Acceptor is silently casting payload of my gziped response-message to string which makes me somehow sad.

Do you know, how to make sure, that the reponse payload won't be touched on the Http Acceptor?

What kind of message type and mime-mapping should I use to make sure, that the compressed payload won't be destroyed because of the internal type transformations?

And again - do you know, if there is a way to configure the embedded jetty and for example apply some http filters in front of http acceptors?

thanks and regards,

maciej

Posted by tsteinbo on 21-Jul-2010 05:50

You should be able to change the content type mapping of the acceptor's protocol handler from TextMessage to BytesMessage to achieve what you want.

Thomas

Posted by Maciej.Barcz@ottogroup.com on 21-Jul-2010 06:12

Hi,

my understanding is, that setting X-JMS-MessageType to bytes I will get Content-Type="application/octet-stream" (evtl. "application/*") + Content-Encoding="gzip" on the HTTP Side.

I'm pretty sure I want Content-Type="text/xml"  + Content-Encoding="gzip" (exactly, as I got it from my backend).

Am I missing something? Or perhaps there is a way to override the mappings of the http reply content type?

thanks and regards,

maciej

Posted by jerem on 21-Jul-2010 08:17

Marciej,

You can configure the content-Type mapping between xqMsg and http response at the acceptor level.

Take a look at the mq_config_manage.pdf (Sonic MQ configuration and management guide) page 280.

It explain how to change the mapping.

Regards

Chan Jeremy

Posted by Maciej.Barcz@ottogroup.com on 21-Jul-2010 09:29

Hi,

ok, thanks; I think I already tested it, anyway will test again.

I'm also going to follow the "Solution S654" (override the content-type settings with a help of “JMS_SonicMQ_XQ.setContentType” header).

regards,

maciej

This thread is closed