Standby MRP needs old log sequence even after restore of incremental backup on standby

SELECT ARCH.THREAD# "Thread", ARCH.SEQUENCE# "Last Sequence Received", APPL.SEQUENCE# "Last Sequence Applied", (ARCH.SEQUENCE# - APPL.SEQUENCE#) "Difference" 
  FROM (SELECT THREAD# ,SEQUENCE# 
  FROM V$ARCHIVED_LOG 
  WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) 
  FROM V$ARCHIVED_LOG 
  GROUP BY THREAD#)) ARCH, 
  (SELECT THREAD# ,SEQUENCE# 
  FROM V$LOG_HISTORY 
  WHERE (THREAD#,FIRST_TIME ) IN (SELECT THREAD#,MAX(FIRST_TIME) 
  FROM V$LOG_HISTORY 
  GROUP BY THREAD#)) APPL 
  WHERE ARCH.THREAD# = APPL.THREAD# 
  ORDER BY 1;


  Thread Last Sequence Received Last Sequence Applied Difference
---------- ---------------------- --------------------- ----------
  1 23861 23057 804
  2 8553 8259 294
  3 8542 8248 294

select process,status,sequence# from v$managed_standby where PROCESS ='MRP0';

PROCESS STATUS SEQUENCE#
--------- ------------ ----------
MRP0 WAIT_FOR_GAP 17353

 Standby db has just been restored from incremental backup taken from primary.

CAUSE
 At least one datafile has scn much lower than the rest of the database:

select distinct to_char(fhscn, '99999999999999999999') as "Stby_datafile_hdr_SCN" from X$KCVFH;

Stby_datafile_hdr_SCN
---------------------
  10571110002345
  10591313115628
  10591313133587
  10591313148383
  10591313156525
  10591313172772
  10591313186729

select min(to_char(fhscn, '99999999999999999999')) as Standby_Datafile_Header_SCN from X$KCVFH;

STANDBY_DATAFILE_HEAD
---------------------
  10571110002345

x$kcbfh.fhscn gives checkpoint scn for a particular datafile. Datafile recovery should start with this checkpoint scn. If scn is too low, the redo containing this could be found in very old archive redo log which may already been deleted.

SOLUTION

a. Identify the datafile with lowest checkpoint scn at standby:
 
col name format a55
select file#, name, to_char(checkpoint_change#, '99999999999999999999') from v$datafile_header where checkpoint_change# < 10571110002350;

  FILE# NAME TO_CHAR(CHECKPOINT_CH
  ---------- ------------------------------------------------------- ---------------------
  299 +DATA_DG1/fsolstby/datafile/sysaux.622.811623929 10571110002345

  b. at primary:

select file#, name from v$datafile where file# = 299 

2. Copy datafile identified in step 1 from primary to standby database
3. Recreate standby control file from current control file at primary. And restore at standby.
4. Restart standby recovery.
alter database recover managed standby database disconnect;