connectivity issue

When you create a connection to the Domain Manager you do that via a JMSConnectorClient which has a messaging level connection to the MQ brokers used for management (i.e the one(s) in the domain manager container(s)).

Since we do not support replicated brokers for management communications, I assume the replicated MQ brokers you want to monitor are in non-DM containers.

The IBrokerProxy creates a binding between a management client and specified individual MQ broker instance, not a replication pair. The binding takes the form of a particular messaging subject space over the management connection.

Example: say you have "ContainerA" hosting "MQPrimary" and "ContainerB" hosting "MQBackup" in domain "Domain1" .. then if you create an IBrokerProxy to "Domain1.ContainerA:ID=MQPrimary" then that proxy is only bound to "Domain1.ContainerA:ID=MQPrimary" .. if you want to look at the state of "MQBackup" you will need to create another proxy to "Domain1.ContainerB:ID=MQBackup".

So your monitoring application is "connected" to the management broker(s), not the primary broker .. but it has a proxy (sitting on top of that connection) that is "bound" to the primary. You probably need to change your application so that if you get a timeout exception from the proxy bound to the primary then you try a proxy to the backup.

Thanks David.

I think, I'm following the same approach you have suggested.

I'm having a appliaction which will send out notifications when flow control happens.Below is the sample code I'm using to get the broker proxy.

Once the broker proxy is created , the appliactions listens to notifications.

try

connector = CommonUtilities.getJMSConnectorClient(url, user, pass);

// create a proxy to the broker component

ObjectName jmxName =

new ObjectName((new StringBuilder())

.append(domain).append(

".").append(primaryContainer)

.append(

":ID=").append(primaryBroker).toString());

broker = MQProxyFactory.createBrokerProxy(connector, jmxName);

catch (Exception e) {

".").append(backupContainer)

.append(

":ID=").append(backupBroker).toString());

broker = MQProxyFactory.createBrokerProxy(connector, jmxName);

}

The problem here is, how the appliaction will know that the primay DS has went down.Is there exists any API method which throws out exception or notify the application to connect to the backup DS automatically.

Thanks,

Habin

Just to pin point the issue,

Suppose a connection is a made via JMSConnectorClient and the DS switch over happens, then the connection is dropped without throwing any exception.Is there a way to handle this scenario ?

-Habin

You can to handle that, reconnect machenism in your JMSConnectorClient if you are writing this.

could you please elaborate on this.I'm not able to understand how this can be implemented.

I am confused by your comments; I thought you were trying to monitor a FT broker pair .. yet you then talk about the primary DS going down?

If you are talking about a FT broker pair then here is a suggestion:

1. Create connector
2. Create an ObjectName to primary
3. Create an ObjectName to backup
4. Create a listener that listens to flow control notifications from both the primary and backup .. only the one that is active can possible send out flow control notifications

I think you are still confused about the components and their purpose in a Sonic deployment .. if you would like we can do a 30min webex where I can go over this with a few diagrams. Look at my calendar to see what might work for us both.

- David

Hi David,
        

    Thanks for the suggestion.
    I'll try to reframe the words once agian so that you can have a better understanding of the problem.

Environment :
The environment have two DS brokers in place.The monitroing appliaction is connecting to these throgh
JMSConnectorClinet using comma seprated URLs.

Problem scenario:
Suppose,I make the primary DS down.All the connections are moved to the secondry DS (as expected) but the
monitoring application conneced using JMSConnectorClinet just go down without throwing any exception
    .They don't connect to the secondry DS.
   
I have some doubts here:

1.   Suppose the application is connected to both primary and secondy directory services as you mentioned.Is
       there any way that the listener application can automatically reconnect if one of the DS goes down ?
2. As url is comma seprated while creating JMSConnectorClient.So I'm expecting when the primary DS goes down, the
   listener application will automatically connect to secondry DS.Am I expecting the correct thing to happen?

Hope the scenario is clear now.

Is there a way I can write reconnect logic or route the connection to the active DS when one DS goes down.

Thanks,
Habin

Habin.. again I think you are getting mixed up on terminolgy and expected behavior:

• The DS does not have brokers; the Domain Manager (DM) is the function of DS, AM and brokers used for management communications .. when we refer to the DM it would be the composite of a container (hosting), the DS, the AM and a management broker .. and if it were a CAA setup then it would be the composite of the containers (hosting), the primary and backup DS, the primary and backup AM and the cluster of management brokers .. and then the primary DM would be the container that includes the primary DS/AM and the backup DM would be the container that includes the backup DS/AM
• The JMSConnectorClient connects to management broker(s) and when provided with a URL list will automatically reconnect itself when the management broker it is connected to goes down
• Once you have added notification listeners to one or more JMX MBeans through our connector, then even if the MBean goes away (e.g. its container is shutdown) the connector will try to maintain those subscriptions so that when the Mbean again becomes available (e.g. it is restarted), the notification listeners will be readded (although there may be some period up to the request timeout before the listeners are actually readded)

I verified the latter behavior by setting up a CAA DM setup, connecting the sample JMX command line with a URL list and then shutting down the primary DM. The connector used by the command line sample reported its failed connection, and did indeed reconnect to the other management broker. Here is some sample output:

davidg@lxdavidg:~/Sonic/8.1/sandboxes/migtest8.1/MQ8.1/samples/Management/runtimeAPI/javaJMX$ sh ../../../Mgmt.sh MgmtCLI tcp://lxdavidg:15000,tcp://lxdavidg:16000 Administrator Administrator
> cc MgmtQADomain
MgmtQADomain> show status

MgmtQADomain.DSContainer (state=Online, host=lxdavidg)
- lxdavidg15000 (state=Online)
- DIRECTORY SERVICE (state=Online)
- AGENT (state=Online)
- AGENT MANAGER (state=Online)

MgmtQADomain.DSContainerBackup (state=Online, host=lxdavidg)
- DIRECTORY SERVICE (state=Online)
- AGENT (state=Online)
- AGENT MANAGER (state=Online)
- lxdavidg16000 (state=Online)

MgmtQADomain>
Disconnected from tcp://lxdavidg:15000,tcp://lxdavidg:16000
MgmtQADomain>
Reconnected to tcp://lxdavidg:15000,tcp://lxdavidg:16000
MgmtQADomain> show status

MgmtQADomain.DSContainer (state=Unknown, host=)

MgmtQADomain.DSContainerBackup (state=Online, host=lxdavidg)
- DIRECTORY SERVICE (state=Online)
- AGENT (state=Online)
- AGENT MANAGER (state=Online)
- lxdavidg16000 (state=Online)

MgmtQADomain>

I even tried adding notification listeners to both the management brokers for connect events and it worked as I expected.

Thanks a lot David.Sorry for the confusing terms .

The scenario was that if one of the DM goes down, the application was not able to connect to the second DM.

As you mentioned, I too tested by adding the notification listener.So, this time the application was able to connect to the second DM.

Test 1 : Drop the connection from smc.

Result : Application was able to connect back.

Test 2 : shut down the management broker.

Result : Appliaction was able to connect to the other management broker.

Test 3 : shut down the DM

Result : Application was able to connect to other DM.

Can this be implemented without adding the listener.I mean the reconnecting logic.

Thanks in advance,

Habin

The example I gave above was without adding a listener (and then mentioned at the end that I also tested after a listener had been added) .. there should be no reconnect logic required in your custom management application.

My specific suggestion was for the custom management application to create a JMX listener on both primary and backup (you can use the same listener and look at any received notification's attributes to find where the notification came from).