Samba Replication Failures

We found new Samba replication errors in the syslog yesterday that apparently had been occurring for a couple weeks. 

Jun 18 21:17:36 dc-3 samba[13322]: [2018/06/18 21:17:36.524971,  0] ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
Jun 18 21:17:36 dc-3 samba[13322]:   ldb: No objectClass found in replPropertyMetaData for DC=DESKTOP-NEW\0ADEL:079138d3-16a0-4806-9b32-0cd694137316,CN=Deleted Objects,DC=DomainDnsZones,DC=acme,DC=com!
Jun 18 21:17:36 dc-3 samba[13322]:
Jun 18 21:17:36 dc-3 samba[13322]: [2018/06/18 21:17:36.525322,  0] ../source4/dsdb/repl/replicated_objects.c:783(dsdb_replicated_objects_commit)
Jun 18 21:17:36 dc-3 samba[13322]:   Failed to apply records: replmd_replicated_apply_add: error during DRS repl ADD: No objectClass found in replPropertyMetaData for DC=DESKTOP-NEW\0ADEL:079138d3-16a0-4806-9b32-0cd694137316,CN=Deleted Objects,DC=DomainDnsZones,DC=acme,DC=com!
Jun 18 21:17:36 dc-3 samba[13322]:   : Object class violation
Jun 18 21:17:36 dc-3 samba[13322]: [2018/06/18 21:17:36.525892,  0] ../source4/dsdb/repl/drepl_out_helpers.c:773(dreplsrv_op_pull_source_apply_changes_trigger)
Jun 18 21:17:36 dc-3 samba[13322]:   Failed to commit objects: WERR_GENERAL_FAILURE/NT_STATUS_INVALID_NETWORK_RESPONSE

The showrepl command was also showing an error on the inbound neighbors DomainDnsZones.

Solution

After further digging we found the 0ADEL records (highlighted above) are records used to indicate when a DNS record has been deleted.  In our case, the additional domain controller with the errors had no reference of the original DNS records so it couldn’t apply the delete.  Not sure how we got into this state.

Found the following blog post which helped us understand how to delete the delete records (its meta).

https://help.univention.com/t/samba-4-deleted-objects/6445

Basically, you can use the ldbsearch and ldbdel commands to find and delete “deleted” records:

sudo ldbsearch -H /var/lib/samba//private/sam.ldb “CN=Desktop*” –show-deleted –cross-ncs

sudo ldbdel -H /var/lib/samba/private/sam.ldb “<GUID=079138d3-16a0-4806-9b32-0cd694137316>” –show-deleted –relax

The ldbsearch command will verify the delete record exists and the ldbdel can delete it.  Notice we use the –show-deleted flag on both and –relax on the delete.  The GUID came from the syslog error message and would also be listed on the ldbsearch output.

I had to do this for several records but this ultimately solved my problem.

Happy Troubleshooting!