All nodes are configured to have quorum votes

Your cluster quorum configuration will be changed to the configuration shown above.

An error was encountered while modifying the quorum settings.
Your cluster quorum settings have not been changed.
There was an error configuring the file share witness '\\dc1\Cluster_Vote'.
Unable to save property changes for 'File Share Witness'.
The system cannot find the file specified

To fix this, add full permission to \\dc1\Cluster_Vote for user Cluster$ (in my case, it’s VirtualDB$)

[Warning Message]

You have specified a file server as the data directory \\dc1\sql2014_DB_SMB\MSSQL12.SQL2014\MSSQL\DATA. To avoid possible failures in the installation process, you must verify that the SQL Server service account has full control share permissions on the specified file server before continuing.

[Details]

Microsoft.SqlServer.Configuration.SqlEngine.ValidationException: You have specified a file server as the data directory \\dc1\sql2014_DB_SMB\MSSQL12.SQL2014\MSSQL\DATA. To avoid possible failures in the installation process, you must verify that the SQL Server service account has full control share permissions on the specified file server before continuing.

Error message:

12:13:08 AM 0x800706BA: The RPC server is unavailable.

12:13:08 AM FS-10751: Node node1 failed to create a class factory on remote node node2

12:13:08 AM FS-10755: Error creating the OracleFsWorker COM object

Fix: Disable the firewall rules between 2 failsafe nodes (netsh advfirewall set allprofiles state off)

Server version:         5.6.19-log MySQL Community Server (GPL)

Original Table Data:

root@server1> select * from appdb.t1;
+----+---------------------+
| id | name                |
+----+---------------------+
|  1 | N1                  |
|  2 | N2a                 |
| 11 | N1                  |
| 21 | 2014-07-20 09:51:53 |
+----+---------------------+
4 rows in set (0.08 sec)

Deletion without matching row ignored silently

root@server1> stop slave io_thread;
root@server2> stop slave io_thread;

root@server1> delete from appdb.t1 where id=1;
root@server2> delete from appdb.t1 where id=1;

root@server1> start slave io_thread;
root@server2> start slave io_thread;

Update executed without checking before image (very different from OGG, Streams, etc)

root@server1> stop slave io_thread;
root@server2> stop slave io_thread;

root@server1> update appdb.t1 set name='server1' where id=2;
root@server2> update appdb.t1 set name='server2' where id=2;

root@server1> start slave io_thread;
root@server2> start slave io_thread;

root@server1> select * from appdb.t1 where id=2;
+----+---------+
| id | name    |
+----+---------+
|  2 | server2 |
+----+---------+
1 row in set (0.00 sec)

root@server2> select * from appdb.t1 where id=2;
+----+---------+
| id | name    |
+----+---------+
|  2 | server1 |
+----+---------+

Install Plugins

root@server1> install plugin rpl_semi_sync_master soname 'semisync_master.so';
root@server1> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';

root@server2> install plugin rpl_semi_sync_master soname 'semisync_master.so';
root@server2> install plugin rpl_semi_sync_slave soname 'semisync_slave.so';

Enable replication

root@server1> set global rpl_semi_sync_master_enabled = 1;
root@server1> set global rpl_semi_sync_master_timeout = 10000;
root@server1> set global rpl_semi_sync_slave_enabled = 1;

root@server2> set global rpl_semi_sync_master_enabled = 1;
root@server2> set global rpl_semi_sync_master_timeout = 10000;
root@server2> set global rpl_semi_sync_slave_enabled = 1;

Restart Slaves

root@server1> stop slave;
root@server1> start slave;

root@server2> stop slave;
root@server2> start slave;

Monitoring the Status

root@server1> show variables like 'rpl_semi_sync%';
+------------------------------------+-------+
| Variable_name                      | Value |
+------------------------------------+-------+
| rpl_semi_sync_master_enabled       | ON    |
| rpl_semi_sync_master_timeout       | 10000 |
| rpl_semi_sync_master_trace_level   | 32    |
| rpl_semi_sync_master_wait_no_slave | ON    |
| rpl_semi_sync_slave_enabled        | ON    |
| rpl_semi_sync_slave_trace_level    | 32    |
+------------------------------------+-------+
6 rows in set (0.00 sec)

root@server1> update appdb.t1 set name='server1' where id=2;

root@server1> show status like 'rpl_semi_sync%';
+--------------------------------------------+----------+
| Variable_name                              | Value    |
+--------------------------------------------+----------+
| Rpl_semi_sync_master_clients               | 1        |
| Rpl_semi_sync_master_net_avg_wait_time     | 2178     |
| Rpl_semi_sync_master_net_wait_time         | 2178     |
| Rpl_semi_sync_master_net_waits             | 1        |
| Rpl_semi_sync_master_no_times              | 1        |
| Rpl_semi_sync_master_no_tx                 | 3        |
| Rpl_semi_sync_master_status                | ON       |
| Rpl_semi_sync_master_timefunc_failures     | 0        |
| Rpl_semi_sync_master_tx_avg_wait_time      | 4011376  |
| Rpl_semi_sync_master_tx_wait_time          | 20056881 |
| Rpl_semi_sync_master_tx_waits              | 5        |
| Rpl_semi_sync_master_wait_pos_backtraverse | 4        |
| Rpl_semi_sync_master_wait_sessions         | 0        |
| Rpl_semi_sync_master_yes_tx                | 1        |
| Rpl_semi_sync_slave_status                 | ON       |
+--------------------------------------------+----------+
15 rows in set (0.01 sec)

root@server1> select * from information_schema.plugins where plugin_name like 'rpl%'\G
*************************** 1. row ***************************
           PLUGIN_NAME: rpl_semi_sync_master
        PLUGIN_VERSION: 1.0
         PLUGIN_STATUS: ACTIVE
           PLUGIN_TYPE: REPLICATION
   PLUGIN_TYPE_VERSION: 2.0
        PLUGIN_LIBRARY: semisync_master.so
PLUGIN_LIBRARY_VERSION: 1.4
         PLUGIN_AUTHOR: He Zhenxing
    PLUGIN_DESCRIPTION: Semi-synchronous replication master
        PLUGIN_LICENSE: GPL
           LOAD_OPTION: ON
*************************** 2. row ***************************
           PLUGIN_NAME: rpl_semi_sync_slave
        PLUGIN_VERSION: 1.0
         PLUGIN_STATUS: ACTIVE
           PLUGIN_TYPE: REPLICATION
   PLUGIN_TYPE_VERSION: 2.0
        PLUGIN_LIBRARY: semisync_slave.so
PLUGIN_LIBRARY_VERSION: 1.4
         PLUGIN_AUTHOR: He Zhenxing
    PLUGIN_DESCRIPTION: Semi-synchronous replication slave
        PLUGIN_LICENSE: GPL
           LOAD_OPTION: ON
2 rows in set (0.00 sec)

What to fix:

2014-07-21 06:29:58 3850 [Warning] Slave SQL: If a crash happens this configuration does not guarantee that the relay log info will be consistent, Error_code: 0

Current Settings

root@server1> select table_name,engine from information_schema.tables
    -> where table_name in ('slave_relay_log_info','slave_master_info');
+----------------------+--------+
| table_name           | engine |
+----------------------+--------+
| slave_master_info    | InnoDB |
| slave_relay_log_info | InnoDB |
+----------------------+--------+
2 rows in set (0.00 sec)

root@server1> show variables like '%info_repository';
+---------------------------+-------+
| Variable_name             | Value |
+---------------------------+-------+
| master_info_repository    | FILE  |
| relay_log_info_repository | FILE  |
+---------------------------+-------+
2 rows in set (0.00 sec)

Add 2 lines to server 1/2 my.cnf

master_info_repository=TABLE
relay_log_info_repository=TABLE

Restart mysqld

Verify the Setting

root@server1> show variables like '%info_repository';
+---------------------------+-------+
| Variable_name             | Value |
+---------------------------+-------+
| master_info_repository    | TABLE |
| relay_log_info_repository | TABLE |
+---------------------------+-------+
2 rows in set (0.00 sec)

root@server1> select * from mysql.slave_master_info \G
*************************** 1. row ***************************
       Number_of_lines: 23
       Master_log_name: bin-log.000008
        Master_log_pos: 120
                  Host: 127.0.0.1
             User_name: repl_user
         User_password: repl_pass
                  Port: 3307
         Connect_retry: 60
           Enabled_ssl: 0
                Ssl_ca:
            Ssl_capath:
              Ssl_cert:
            Ssl_cipher:
               Ssl_key:
Ssl_verify_server_cert: 0
             Heartbeat: 1800
                  Bind:
    Ignored_server_ids: 0
                  Uuid: 37f99013-0fa4-11e4-ae2f-e13c88f43118
           Retry_count: 86400
               Ssl_crl:
           Ssl_crlpath:
Enabled_auto_position: 0
1 row in set (0.00 sec)

root@server1> select * from mysql.slave_relay_log_info \G
*************************** 1. row ***************************
  Number_of_lines: 7
   Relay_log_name: /mysql56/mysql-server1/slave/relay-bin.000021
    Relay_log_pos: 281
  Master_log_name: bin-log.000008
   Master_log_pos: 120
        Sql_delay: 0
Number_of_workers: 0
               Id: 1
1 row in set (0.00 sec)

Step-by-step instructions:

Install the Microsoft Baseline Configuration Analyzer v2 (BCA) – This will install to C:\Program Files\Microsoft Baseline Configuration Analyzer 2\.  (http://www.microsoft.com/en-sg/download/details.aspx?id=16475)

Install the Microsoft SQL Server 2012 Best Practices Analyzer (SQL BPA)– This will install to C:\Windows\System32\BestPractices\v1.0\Models\SQL2012BPA\. (http://www.microsoft.com/en-sg/download/details.aspx?id=29302)

Navigate to the directory C:\Windows\System32\BestPractices\v1.0\Models and right-click the SQL2012BPA folder and make a copy.

Navigate to C:\ProgramData\Microsoft\Microsoft Baseline Configuration Analyzer 2\Models\ and paste the SQL2012BPA folder in this new location.

Start the BCA with Run As Administrator and you will now see the SQL Server 2012 BPA listed in the “Select a product” dropdown.

[oracle@rac1 ~]$ crsctl replace -h
Usage:
  crsctl replace {discoverystring <ds_string>| votedisk [<+diskgroup>|<vdisk> ... <vdisk>]}
where
    ds_string   comma-separated voting file paths without spaces and enclosed in quotes
    diskgroup   diskgroup where the voting files will be located in ASM
    vdisk       location of the voting files outside of ASM separated by space

SQL> select * from v$version;

BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
PL/SQL Release 11.2.0.4.0 - Production
CORE    11.2.0.4.0      Production
TNS for Linux: Version 11.2.0.4.0 - Production
NLSRTL Version 11.2.0.4.0 - Production

[oracle@rac1 ~]$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
1. ONLINE   5df885510a334fe3bfb7716a32bc0511 (/dev/sdb1) [OCRVOTE]
Located 1 voting disk(s).

SQL> create diskgroup ocr_vote normal redundancy
  2  failgroup site1 disk '/dev/sde1'
  3  failgroup site2 disk '/dev/sdf1'
  4  quorum failgroup site3 disk '/dev/sdg1'
  5  attribute 'compatible.asm' = '11.2.0.0.0';

Diskgroup created.

[oracle@rac1 ~]$ crsctl replace votedisk +ocr_vote
Successful addition of voting disk a24090f028f84f0bbff82d389fbfc140.
Successful addition of voting disk 24fd01904c504f7cbf4d8478a37d15b9.
Successful addition of voting disk a1ef7dd357064f85bf6a11cbf508ebb6.
Successful deletion of voting disk 5df885510a334fe3bfb7716a32bc0511.
Successfully replaced voting disk group with +ocr_vote.
CRS-4266: Voting file(s) successfully replaced

[oracle@rac1 ~]$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
1. ONLINE   a24090f028f84f0bbff82d389fbfc140 (/dev/sdg1) [OCR_VOTE]
2. ONLINE   24fd01904c504f7cbf4d8478a37d15b9 (/dev/sde1) [OCR_VOTE]
3. ONLINE   a1ef7dd357064f85bf6a11cbf508ebb6 (/dev/sdf1) [OCR_VOTE]
Located 3 voting disk(s).

Error message from /u01/app/11.2.0/grid/log/rac1/alertrac1.log

2014-07-27 00:39:04.870:
[cssd(2446)]CRS-1637:Unable to locate configured voting file with ID a24090f0-28f84f0b-bff82d38-9fbfc140; details at (:CSSNM00020:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2014-07-27 00:39:04.870:
[cssd(2446)]CRS-1637:Unable to locate configured voting file with ID a1ef7dd3-57064f85-bf6a11cb-f508ebb6; details at (:CSSNM00020:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2014-07-27 00:39:04.870:
[cssd(2446)]CRS-1705:Found 1 configured voting files but 2 voting files are required, terminating to ensure data integrity; details at (:CSSNM00021:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2014-07-27 00:39:04.870:
[cssd(2446)]CRS-1656:The CSS daemon is terminating due to a fatal error; Details at (:CSSSC00012:) in /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
2014-07-27 00:39:04.910:
[cssd(2446)]CRS-1603:CSSD on node rac1 shutdown by user.

Error message from /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log

2014-07-27 00:39:12.734: [    CSSD][3380029184]clssnmvDiskVerify: discovered a potential voting file
2014-07-27 00:39:12.734: [   SKGFD][3380029184]Handle 0x7f5ab40967a0 from lib :UFS:: for disk :/dev/sde1:

2014-07-27 00:39:12.738: [    CSSD][3380029184]clssnmvDiskVerify: Successful discovery for disk /dev/sde1, UID 24fd0190-4c504f7c-bf4d8478-a37d15b9, Pending CIN 0:1406391959:
0, Committed CIN 0:1406391959:0
2014-07-27 00:39:12.738: [   SKGFD][3380029184]Lib :UFS:: closing handle 0x7f5ab40967a0 for disk :/dev/sde1:

2014-07-27 00:39:12.738: [    CSSD][3380029184]clssnmvDiskVerify: Successful discovery of 1 disks
2014-07-27 00:39:12.738: [    CSSD][3380029184]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2014-07-27 00:39:12.738: [    CSSD][3380029184]clssnmCompleteVFDiscovery: Completing voting file discovery
2014-07-27 00:39:12.738: [    CSSD][3380029184]clssnmvDiskStateChange: state from discovered to pending disk /dev/sde1
2014-07-27 00:39:12.738: [    CSSD][3380029184]clssnmvDiskStateChange: state from pending to configured disk /dev/sde1
2014-07-27 00:39:12.738: [    CSSD][3380029184]clssnmvVerifyCommittedConfigVFs: Insufficient voting files found, found 1 of 3 configured, needed 2 voting files
2014-07-27 00:39:12.738: [    CSSD][3380029184](:CSSNM00020:)clssnmvVerifyCommittedConfigVFs: voting file 0, id a24090f0-28f84f0b-bff82d38-9fbfc140 not found
2014-07-27 00:39:12.738: [    CSSD][3380029184](:CSSNM00020:)clssnmvVerifyCommittedConfigVFs: voting file 2, id a1ef7dd3-57064f85-bf6a11cb-f508ebb6 not found
2014-07-27 00:39:12.738: [    CSSD][3380029184]ASSERT clssnm1.c 3336
2014-07-27 00:39:12.738: [    CSSD][3380029184](:CSSNM00021:)clssnmCompleteVFDiscovery: Found 1 voting files, but 2 are required.  Terminating due to insufficient configured
voting files
2014-07-27 00:39:12.738: [    CSSD][3380029184]###################################
2014-07-27 00:39:12.738: [    CSSD][3380029184]clssscExit: CSSD aborting from thread clssnmvDDiscThread
2014-07-27 00:39:12.738: [    CSSD][3380029184]###################################
2014-07-27 00:39:12.738: [    CSSD][3380029184](:CSSSC00012:)clssscExit: A fatal error occurred and the CSS daemon is terminating abnormally

How to fix it:

[root@rac1 ~]# crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'rac1'
CRS-2673: Attempting to stop 'ora.crf' on 'rac1'
CRS-2677: Stop of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.

[root@rac1 ~]# crsctl start crs -excl
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac1'
CRS-2676: Start of 'ora.mdnsd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac1'
CRS-2676: Start of 'ora.gpnpd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac1'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac1'
CRS-2676: Start of 'ora.diskmon' on 'rac1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac1'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2676: Start of 'ora.ctssd' on 'rac1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac1'
CRS-2676: Start of 'ora.asm' on 'rac1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac1'
CRS-2676: Start of 'ora.crsd' on 'rac1' succeeded

[root@rac1 ~]# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
1. OFFLINE  a24090f028f84f0bbff82d389fbfc140 () []
2. ONLINE   24fd01904c504f7cbf4d8478a37d15b9 (/dev/sde1) [OCR_VOTE]
3. OFFLINE  a1ef7dd357064f85bf6a11cbf508ebb6 () []
Located 3 voting disk(s).

[root@rac1 ~]#  kfod op=groups
--------------------------------------------------------------------------------
Group          Size          Free Redundancy Name
================================================================================
   1:       1023 Mb        659 Mb     EXTERN OCRVOTE
   2:      30719 Mb      28516 Mb     EXTERN DATA
  

[root@rac1 ~]# crsctl replace votedisk +OCRVOTE
Successful addition of voting disk 6d1dc365ec824f94bfa7fc18f8e3f6b0.
Successful deletion of voting disk a24090f028f84f0bbff82d389fbfc140.
Successful deletion of voting disk 24fd01904c504f7cbf4d8478a37d15b9.
Successful deletion of voting disk a1ef7dd357064f85bf6a11cbf508ebb6.
Successfully replaced voting disk group with +OCRVOTE.
CRS-4266: Voting file(s) successfully replaced
 

[root@rac1 ~]# crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'
CRS-2673: Attempting to stop 'ora.asm' on 'rac1'
CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'
CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'
CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'
CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.

[root@rac1 ~]# crsctl start crs

CRS-4123: Oracle High Availability Services has been started.

[root@rac1 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       rac1
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac1
ora.OCRVOTE.dg
               ONLINE  ONLINE       rac1
ora.OCR_VOTE.dg
               ONLINE  OFFLINE      rac1
ora.asm
               ONLINE  ONLINE       rac1                     Started
ora.gsd
               OFFLINE OFFLINE      rac1
ora.net1.network
               ONLINE  ONLINE       rac1
ora.ons
               ONLINE  ONLINE       rac1
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac1
ora.cvu
      1        ONLINE  ONLINE       rac1
ora.oc4j
      1        ONLINE  ONLINE       rac1
ora.orcl.db
      1        ONLINE  ONLINE       rac1                     Open
      2        ONLINE  OFFLINE
ora.rac1.vip
      1        ONLINE  ONLINE       rac1
ora.rac2.vip
      1        ONLINE  INTERMEDIATE rac1                     FAILED OVER
ora.scan1.vip
      1        ONLINE  ONLINE       rac1

Symptom:

[oracle@rac1 grid]$ crsctl replace votedisk +ocr_vote;
Failed to create voting files on disk group ocr_vote.
Change to configuration failed, but was successfully rolled back.
CRS-4000: Command Replace failed, or completed with errors.

Cause 1: compatible.asm  is default and less than 11.2.0.0.0

Sun Jul 27 01:56:20 2014
NOTE: Creating voting files in diskgroup OCR_VOTE
Sun Jul 27 01:56:20 2014
NOTE: Voting File refresh pending for group 3/0x6e9432c (OCR_VOTE)
NOTE: Attempting voting file creation in diskgroup OCR_VOTE
ERROR: Voting file allocation failed for group OCR_VOTE
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_ora_5279.trc:
ORA-15221: ASM operation requires compatible.asm of 11.2.0.0.0 or higher
NOTE: Attempting voting file refresh on diskgroup OCR_VOTE
NOTE: Refresh completed on diskgroup OCR_VOTE. No voting file found.

Cause 2: not enough failgroups to support required copy for redundancy

Sun Jul 27 02:06:52 2014
NOTE: Creating voting files in diskgroup OCR_VOTE
Sun Jul 27 02:06:52 2014
NOTE: Voting File refresh pending for group 3/0x71294335 (OCR_VOTE)
NOTE: Attempting voting file creation in diskgroup OCR_VOTE
NOTE: voting file allocation on grp 3 disk OCR_VOTE_0000
NOTE: voting file allocation on grp 3 disk OCR_VOTE_0001
ERROR: Voting file allocation failed for group OCR_VOTE
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_ora_6738.trc:
ORA-15274: Not enough failgroups (3) to create voting files
NOTE: Attempting voting file refresh on diskgroup OCR_VOTE
NOTE: Refresh completed on diskgroup OCR_VOTE. No voting file found.
NOTE: Voting file relocation is required in diskgroup OCR_VOTE
NOTE: Attempting voting file relocation on diskgroup OCR_VOTE
NOTE: voting file deletion on grp 3 disk OCR_VOTE_0000
NOTE: voting file deletion on grp 3 disk OCR_VOTE_0001
NOTE: No voting file found on diskgroup OCR_VOTE

[oracle@rac1 trace]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

[oracle@rac1 trace]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2920
         Available space (kbytes) :     259200
         ID                       : 1238456451
         Device/File Name         :  +OCRVOTE
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check bypassed due to non-privileged user

[oracle@rac1 trace]$ ocrconfig -replace +OCRVOTE -replacement +OCR_VOTE
PROT-20: Insufficient permission to proceed. Require privileged user

[root@rac1 ~]# ocrconfig -replace +OCRVOTE -replacement +OCR_VOTE
PROT-28: Cannot delete or replace the only configured Oracle Cluster Registry location

[root@rac1 ~]# ocrconfig -add +OCR_VOTE

[root@rac1 ~]# ocrconfig -delete +OCRVOTE

[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2920
         Available space (kbytes) :     259200
         ID                       : 1238456451
         Device/File Name         : +OCR_VOTE
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

Noticed that standard edition is grey out, after further research, found below note:

Oracle Database 12c Release 1 Patchset 1 (12.1.0.2) is being released only as Enterprise Edition at this time. At some later date, we expect to release 12.1.0.2 Standard Edition and Standard Edition One. The Installer in the initial release will only offer the option of installing as Enterprise Edition, and licensed use will require an Enterprise Edition license.

oracle@solaris:/u01/stage/database$ export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1

oracle@solaris:/u01/stage/database$ export PATH=/u01/app/oracle/product/12.1.0.2/dbhome_1/bin:/usr/bin:/usr/sbin:/usr/local/bin

oracle@solaris:/u01/stage/database$ which dbua

/u01/app/oracle/product/12.1.0.2/dbhome_1/bin/dbua

oracle@solaris:/u01/stage/database$ dbua

Step 1: Create user accounts "pa_admin" to perform privilege analysis

Step 2: Create and enable a privilege analysis policy

2.1 Connect as user pa_admin.

2.2 Create the following privilege analysis policy:

2.3 Enable the policy.

Step 3: Run some dummy workload to simulate activities as user "Donghua"

Step 4: Disable the privilege analysis policy

4.1 Connect as user pa_admin.

4.2 Disable the dba_donghua_priv_analysis_pol privilege policy.

Step 5: Generate and view privilege analysis reports

5.1 As user pa_admin, generate the privilege analysis results.

5.2 Find the system privileges and roles that user "Donghua" used during the privilege analysis period.

5.3 Find the object privileges and roles that user "Donghua" used during the privilege analysis period.

5.4 Find the unused system privileges for user "Donghua".

Step 6: Remove the components for this example

6.1 As user pa_admin, drop the dba_donghua_priv_analysis_pol privilege analysis policy.

6.2 Drop the user pa_admin (connect user donghua)

Without Redaction:

SQL> conn donghua/password       
SQL> create user hr_viewer identified by password;
SQL> grant create session to hr_viewer;
SQL> grant select on hr.employees to hr_viewer;
   
SQL> conn hr_viewer/password
Connected.
SQL>
SQL> select employee_id,salary,phone_number from hr.employees where employee_id in (107,108);

EMPLOYEE_ID     SALARY PHONE_NUMBER
----------- ---------- ------------------------------
        107       4200 590.423.5567
        108      12008 515.124.4569
 

Create policy on column hr.employees.phone_number   

SQL> conn donghua/password   
SQL> BEGIN
  2   DBMS_REDACT.ADD_POLICY(
  3     object_schema          => 'hr',
  4     object_name            => 'employees',
  5     column_name            => 'phone_number',
  6     policy_name            => 'redact_hr_employees',
  7     function_type          => DBMS_REDACT.REGEXP,
  8     function_parameters    => NULL,
  9     expression             => '1=1',
10     regexp_pattern         => '(\d\d\d).(\d\d)(\w+)',
11     regexp_replace_string  => 'XXX.XX\3',
12     regexp_position        => DBMS_REDACT.RE_BEGINNING,
13     regexp_occurrence      => DBMS_REDACT.RE_FIRST,
14     regexp_match_parameter => DBMS_REDACT.RE_CASE_INSENSITIVE,
15     policy_description     => 'Regular expressions to redact the first 5 digits of U.S. telephone numbers',
16     column_description     => 'phone_number contains employee actual phone number');
17  END;
18  /
 
SQL> conn hr_viewer/password

SQL> col phone_number for a20
SQL> select employee_id,salary,phone_number from hr.employees where employee_id in (107,108);

EMPLOYEE_ID     SALARY PHONE_NUMBER
----------- ---------- --------------------
        107       4200 XXX.XX3.5567
        108      12008 XXX.XX4.4569

Add additional column hr.employees.salary to the redaction column list

       
SQL> conn donghua/password   
SQL> BEGIN
  2   DBMS_REDACT.ALTER_POLICY(
  3     object_schema          => 'hr',
  4     object_name            => 'employees',
  5     policy_name            => 'redact_hr_employees',
  6     action                 => DBMS_REDACT.ADD_COLUMN,
  7     column_name            => 'salary',
  8     function_type          => DBMS_REDACT.RANDOM,
  9     policy_description     => 'Regular expressions to redact the first 5 digits of U.S. telephone numbers, and salary column',
10     column_description     => 'Salary information is sensitive');
11  END;
12  /

SQL> conn hr_viewer/password
Connected.
SQL>  col phone_number for a20
SQL> select employee_id,salary,phone_number from hr.employees where employee_id in (107,108);

EMPLOYEE_ID     SALARY PHONE_NUMBER
----------- ---------- --------------------
        107       1246 XXX.XX3.5567
        108       1336 XXX.XX4.4569

SQL> select employee_id,salary,phone_number from hr.employees where employee_id in (107,108);

EMPLOYEE_ID     SALARY PHONE_NUMBER
----------- ---------- --------------------
        107       3090 XXX.XX3.5567
        108       2514 XXX.XX4.4569

Schema owner without “exempt redaction policy” only see the redacted data

SQL> conn hr/password
Connected.
SQL> select employee_id,salary,phone_number from hr.employees where employee_id in (107,108);

EMPLOYEE_ID     SALARY PHONE_NUMBER
----------- ---------- --------------------
        107       1380 XXX.XX3.5567
        108      10640 XXX.XX4.4569

SQL> create table employees_bak as select * from employees;
create table employees_bak as select * from employees
                                     *
ERROR at line 1:
ORA-28081: Insufficient privileges - the command references a redacted object.

Query the dictionary

SQL> conn donghua/password   

SQL> select object_owner||','||object_name||','||policy_name||','||expression||','||enable||','||policy_description
  2  from redaction_policies;

OBJECT_OWNER||','||OBJECT_NAME||','||POLICY_NAME||','||EXPRESSION||','||ENABLE||
--------------------------------------------------------------------------------
HR,EMPLOYEES,redact_hr_employees,1=1,YES    ,Regular expressions to redact the f
irst 5 digits of U.S. telephone numbers
       

SQL> select object_owner||','||object_name||','||column_name||','||function_type||','||function_parameters||','||
  2  regexp_pattern||','||regexp_replace_string||','||regexp_position||','||regexp_occurrence||','
  3  ||regexp_match_parameter||','||column_description
  4  from redaction_columns;

OBJECT_OWNER||','||OBJECT_NAME||','||COLUMN_NAME||','||FUNCTION_TYPE||','||FUNCT
--------------------------------------------------------------------------------
HR,EMPLOYEES,PHONE_NUMBER,REGEXP REDACTION,,(\d\d\d).(\d\d)(\w+),XXX.XX\3,1,1,i,
phone_number contains employee actual phone number
HR,EMPLOYEES,SALARY,RANDOM REDACTION,,,,0,0,,

Working with “exempt redaction policy” privilege

SQL> grant exempt redaction policy to hr_viewer;

Grant succeeded.

SQL> conn hr_viewer/password
Connected.
SQL> col phone_number for a20
SQL> select employee_id,salary,phone_number from hr.employees where employee_id in (107,108);

EMPLOYEE_ID     SALARY PHONE_NUMBER
----------- ---------- --------------------
        107       4200 590.423.5567
        108      12008 515.124.4569

Disable and drop the policy

SQL> conn donghua/password       
           
SQL> BEGIN
  2   DBMS_REDACT.DISABLE_POLICY(
  3     object_schema          => 'hr',
  4     object_name            => 'employees',
  5     policy_name            => 'redact_hr_employees');
  6  END;
  7  /

PL/SQL procedure successfully completed.

SQL>
SQL> BEGIN
  2   DBMS_REDACT.DROP_POLICY(
  3     object_schema          => 'hr',
  4     object_name            => 'employees',
  5     policy_name            => 'redact_hr_employees');
  6  END;
  7  /

PL/SQL procedure successfully completed.

oracle@solaris:~$ sqlplus donghua@orcl

SQL*Plus: Release 12.1.0.2.0 Production on Mon Aug 4 07:21:18 2014

Copyright (c) 1982, 2014, Oracle.  All rights reserved.

Enter password:
Last Successful login time: Mon Aug 04 2014 07:19:37 +08:00

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> create tablespace fbda_1y datafile '/u01/app/oracle/oradata/orcl/fbda_1y.dbf' size 10M autoextend on;

Tablespace created.

SQL> create flashback archive FBDA1 tablespace fbda_1y quota 1G retention 1 year;

Flashback archive created.

SQL> create table emp1 tablespace users flashback archive FBDA1
  2  as select * from hr.employees where 1=2;

Table created.

SQL> alter flashback archive FBDA1 set default;
alter flashback archive FBDA1 set default
*
ERROR at line 1:
ORA-55611: No privilege to manage default Flashback Archive

SQL> conn / as sysdba
Connected.
SQL> alter flashback archive FBDA1 set default;

Flashback archive altered.

SQL> conn donghua
Enter password:
Connected.
SQL>

SQL> col flashback_archive_name for a30
SQL> select owner_name, flashback_archive_name, status
  2  from dba_flashback_archive;

OWNER_NAME                     FLASHBACK_ARCHIVE_NAME         STATUS
------------------------------ ------------------------------ -------
DONGHUA                        FBDA1                          DEFAULT

SQL> select * from dba_flashback_archive_ts;

FLASHBACK_ARCHIVE_NAME  FLASHBACK_ARCHIVE# TABLESPACE_NAME  QUOTA_IN_MB
----------------------- ------------------ ---------------- -------------
FBDA1                                    1 FBDA_1Y          1024

SQL> select * from dba_flashback_archive_tables;

TABLE_NAME           OWNER_NAME FLASHBACK_ARCHIVE_NAME  ARCHIVE_TABLE_NAME  STATUS
-------------------- ---------- ----------------------- ------------------- --------
EMP1                 DONGHUA    FBDA1                   SYS_FBA_HIST_109887 ENABLED

SQL> insert into emp1
  2  select * from hr.employees where employee_id=107;

1 row created.

SQL> commit;

Commit complete.

SQL> select first_name,last_name from emp1 where employee_id=107;

FIRST_NAME           LAST_NAME
-------------------- -------------------------
Diana                Lorentz

SQL> truncate table emp1;

Table truncated.

SQL> select * from emp1;

no rows selected

SQL> select * from emp1 versions between scn minvalue AND maxvalue;

EMPLOYEE_ID FIRST_NAME           LAST_NAME                 EMAIL                     PHONE_NUMBER         HIRE_DATE
----------- -------------------- ------------------------- ------------------------- -------------------- ---------
JOB_ID         SALARY COMMISSION_PCT MANAGER_ID DEPARTMENT_ID
---------- ---------- -------------- ---------- -------------
        107 Diana                Lorentz                   DLORENTZ                  590.423.5567         07-FEB-07
IT_PROG          4200                       103            60

SQL> drop table emp1;
drop table emp1
           *
ERROR at line 1:
ORA-55610: Invalid DDL statement on history-tracked table

SQL> alter table emp1 drop column email;
SQL> alter table emp1 drop column hire_date;
SQL> alter table emp1 drop column job_id;
SQL> alter table emp1 drop column salary;
SQL> alter table emp1 drop column commission_pct;
SQL> alter table emp1 drop column manager_id;
SQL> alter table emp1 drop column department_id;
SQL> alter table emp1 add (note varchar2(5));

SQL> insert into emp1 values(1,'Donghua','Luo','12c');

1 row created.

SQL> commit;

Commit complete.

SQL> select * from emp1;

EMPLOYEE_ID FIRST_NAME           LAST_NAME                 NOTE
----------- -------------------- ------------------------- -----
          1 Donghua              Luo                       12c

         
SQL> select * from emp1 versions between scn minvalue AND maxvalue;

EMPLOYEE_ID FIRST_NAME           LAST_NAME                 NOTE  D_4956408_EMAIL           D_4956474_PHONE_NUMB
----------- -------------------- ------------------------- ----- ------------------------- --------------------
D_4956528 D_4956605_ D_4956645_SALARY D_4956689_COMMISSION_PCT D_4956735_MANAGER_ID D_4956780_DEPARTMENT_ID
--------- ---------- ---------------- ------------------------ -------------------- -----------------------
        107 Diana                Lorentz                         DLORENTZ                  590.423.5567
07-FEB-07 IT_PROG                4200                                           103                      60

          1 Donghua              Luo                       12c

What is the result of executing a TRUNCATE TABLE command on a table that has Flashback Archiving enabled? Oracle is smart enough to archive the data before executing the truncation on the table.

root@solaris:~# pkg info entire
          Name: entire
       Summary: Incorporation to lock all system packages to the same build
   Description: This package constrains system package versions to the same
                build.  WARNING: Proper system update and correct package
                selection depend on the presence of this incorporation.
                Removing this package will result in an unsupported system.
      Category: Meta Packages/Incorporations
         State: Installed
     Publisher: solaris
       Version: 0.5.11
Build Release: 5.11
        Branch: 0.175.1.0.0.24.2
Packaging Date: September 19, 2012 07:01:35 PM
          Size: 5.46 kB
          FMRI: pkg://solaris/entire@0.5.11,5.11-0.175.1.0.0.24.2:20120919T190135Z

root@solaris:~#   pkg publisher
PUBLISHER                   TYPE     STATUS P LOCATION
solaris                     origin   online Fhttp://pkg.oracle.com/solaris/release/

root@solaris:~# pkg update --accept
------------------------------------------------------------
Package: pkg://solaris/consolidation/osnet/osnet-incorporation@0.5.11,5.11-0.175.2.0.0.42.2:20140624T183842Z
License: lic_OTN

<<<<<<<<<<<<<<<<<<< Ommited length license agreement >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Entire Agreement

You agree that this agreement is the complete agreement for the
Programs and licenses, and this agreement supersedes all prior or
contemporaneous agreements or representations. Any other license terms
that may be included in or with the Program shall not apply.  If any
term of this agreement is found to be invalid or unenforceable, the
remaining provisions will remain effective.

Last updated:  12 May 2014
Should you have any questions concerning this License Agreement, or if
you desire to contact Oracle for any reason, please write:

Oracle America, Inc.
500 Oracle Parkway,
Redwood City, CA 94065

Oracle may contact you to ask if you had a satisfactory experience
installing and using this OTN software download.

            Packages to remove:   6
           Packages to install:  85
            Packages to update: 515
           Mediators to change:   2
       Create boot environment: Yes
Create backup boot environment:  No

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                            606/606   39003/39003  633.4/633.4  161k/s

PHASE                                          ITEMS
Removing old actions                       7730/7730
Installing new actions                   27495/27495
Updating modified actions                28698/28934
driver (cmdk) upgrade (addition of policy 'tpd_member=true') failed with return code 1
command run was: /usr/sbin/update_drv -b /tmp/tmptiazWh -a -p tpd_member=true cmdk
command output was:
------------------------------------------------------------
Bad policy token: ``tpd_member''.
------------------------------------------------------------
Updating modified actions                28701/28934
driver (fp) upgrade (addition of policy 'tpd_member=true') failed with return code 1
command run was: /usr/sbin/update_drv -b /tmp/tmptiazWh -a -p tpd_member=true fp
command output was:
------------------------------------------------------------
Bad policy token: ``tpd_member''.
------------------------------------------------------------
Updating modified actions                28710/28934
driver (sd) upgrade (addition of policy 'tpd_member=true') failed with return code 1
command run was: /usr/sbin/update_drv -b /tmp/tmptiazWh -a -p tpd_member=true sd
command output was:
------------------------------------------------------------
Bad policy token: ``tpd_member''.
------------------------------------------------------------
Updating modified actions                28712/28934
driver (sgen) upgrade (addition of policy 'tpd_member=true') failed with return code 1
command run was: /usr/sbin/update_drv -b /tmp/tmptiazWh -a -p tpd_member=true sgen
command output was:
------------------------------------------------------------
Bad policy token: ``tpd_member''.
------------------------------------------------------------
Updating modified actions                28934/28934
Updating package state database                 Done
Updating package cache                       521/521
Updating image state                            Done
Creating fast lookup database                   Done

A clone of solaris exists and has been updated and activated.
On the next boot the Boot Environment solaris-1 will be
mounted on '/'.  Reboot when ready to switch to this updated BE.

The following unexpected or editable files and directories were
salvaged while executing the requested package operation; they
have been moved to the displayed location in the image:

  usr/lib/python2.6/vendor-packages/solaris_install/auto_install -> /tmp/tmptiazWh/var/pkg/lost+found/usr/lib/python2.6/vendor-packages/solaris_install/auto_install-20140804T091641Z

---------------------------------------------------------------------------
NOTE: Please review release notes posted at:

http://www.oracle.com/pls/topic/lookup?ctx=E26502&id=SERNS
---------------------------------------------------------------------------

root@solaris:~# init 6

root@solaris:~# uname -a
SunOS solaris 5.11 11.2 i86pc i386 i86pc

root@solaris:~# uname -X
System = SunOS
Node = solaris
Release = 5.11
KernelID = 11.2
Machine = i86pc
BusType = <unknown>
Serial = <unknown>
Users = <unknown>
OEM# = 0
Origin# = 1
NumCPU = 2

root@solaris:~# beadm list
BE        Active Mountpoint Space  Policy Created
--        ------ ---------- -----  ------ -------
solaris   -      -          9.61M  static 2012-11-19 19:56
solaris-1 NR     /          21.94G static 2014-08-04 09:16

root@solaris:~# pkg info entire
          Name: entire
       Summary: Incorporation to lock all system packages to the same build
   Description: This package constrains system package versions to the same
                build.  WARNING: Proper system update and correct package
                selection depend on the presence of this incorporation.
                Removing this package will result in an unsupported system.
      Category: Meta Packages/Incorporations
         State: Installed
     Publisher: solaris
       Version: 0.5.11
Build Release: 5.11
        Branch: 0.175.2.0.0.42.0
Packaging Date: June 24, 2014 07:38:32 PM
          Size: 5.46 kB
          FMRI: pkg://solaris/entire@0.5.11,5.11-0.175.2.0.0.42.0:20140624T193832Z

Create a PDB by using the seed

Create a PDB in a CDB using the files of the seed. This technique copies the files associated with the seed to a new location and associates the copied files with the new PDB.

Create a PDB by cloning an existing PDB or non-CDB

Create a PDB by cloning a source PDB or non-CDB and plugging the clone into the CDB. A source can be a PDB in the local CDB, a PDB in a remote CDB, or a non-CDB. This technique copies the files associated with the source to a new location and associates the copied files with the new PDB.

Create a PDB by plugging an unplugged PDB into a CDB

Create a PDB by using the XML metadata file that describes the PDB and the files associated with the PDB to plug it into the CDB.

Create a PDB by using a non-CDB

Create a PDB by moving a non-CDB into a PDB. You can use the DBMS_PDB package to create an unplugged PDB from an Oracle Database 12c non-CDB. You can then plug the unplugged PDB into the CDB.

http://docs.oracle.com/database/121/ADMIN/cdb_plug.htm#ADMIN13551