OEM12C Deployment and Mass Promotion

OEM12C Deployment and Mass Promotion

I was privileged to be a speaker at the Rocky Mountain training days #td17 on the topic of “Automagically” Deploying EM12c Agents and Mass promotion of targets once the hosts have been added.

The presentation is attached, but in a nut shell, using EM12C’s RPM generation and using Puppet we are able to push out agents, the moment the machines are built and validated.

The first step is to log into the OMS machine and generate the RPM using EMCLI (After logging into emcli )

> emcli get_supported_platforms  <== This will get the information for building the RPM’s


emcli get_agentimage_rpm -destination=/tmp -platform=”Linux x86-64″ -version=”″

This last command will generate a RPM in the /tmp directory called: “oracle-agt-”   << notice the version # and platform in the name

Now that the RPM is ready, we use Puppet to copy and to install the RPM on new machines.

A few notes about puppet installs with RPMS

  • We use a standard location / directory
  • Only run if the standard location is NOT in place
  • Only run if “emwd.pl” is not in place
  • Verifies the correct ownership of the directories
  • Uses oraInventory for location

There are 2 steps for the puppet process

  1. apply RPM – >> rpm -ivh oracle-agt-
  2. create agent.properties file – With host name, default password, etc.
  3. run – /etc/init.d/oracle-agt RESPONSE_FILE=/usr/lib/oracle/agent/agent.properties
  4. When Step 3 finishes, there should be a new OMS entry for that host and connected to the OMS.

Last step – Just added recently – (Not part of presentation) is to perform a mass promotion of non-host targets like databases, listeners, etc.


Mass Promote code is in GIT Hub – https://github.com/harry2040/OEM12cMassPromote

Thanks again to #td17 for a great conference and the opportunity.


ORA-14400: inserted partition key does not map to any partition

This occured during our production cloud control upgrade to release 3. After it failed, i did a flashback database and got this same error with the SYSMAN user login.

Found the error and fix in metalink – 1493151.1

Reason for Failing:

A partition was missing from sysman.mgmt_audit table and no audit was being performed. So this caused issues with logging into sysman and upgrade.

to Fix (From metalink) – it works and now the upgrade is completed:

Case 1:

This is an upgrade specific issue. The mgmt_audit_master.prepopulate_days table is not initialized during upgrade, so new partitions are not being created.

job_queue_processes was left at 0 after the upgrading to

Set the job_queue_processes to a non-zero value, then verify the console login.

Case 2:

If the job_queue_processes value is already set to a value greater than 0:

1. Execute the following queries in repository DB:
SQL> update mgmt_audit_master set prepopulate_days=5 where prepopulate_days is null;
SQL> exec mgmt_audit_admin.add_audit_partition;

2. Restart the OMS:

./emctl stop oms -all

./emctl start oms

EM12C – Clearing stateless events

In our system we usually get 40 – 50 stateless incidents due to ora-00060 errors (Poorly written software) type issues.  In the past it has been very difficult to remove them via the GUI console.   I found an article in http://www.freelists.org that helped me fix them – “www.freelists.org/post/oracle-l/Clearing-stateless-events-in-OEM-12c,2“.   There is an automated job (Out of the box) to clean them up, and i will be setting this up in my environment.   Also, the event for alert logs can be modified, but Im still looking at ways to do that in an easy manner.

Here is a snippet of the code to clear the events:

[oracle@machinename bin]$ ./emcli login -username=sysman
Enter password

Login successful
[oracle@machinename bin]$ ./emcli clear_stateless_alerts -older_than=0 -target_type=oracle_database -target_name=<databasename>

 46 alerts were cleared successfully.

[oracle@machine bin]$ ./emcli logout
Logout successful