<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none"><!-- p { margin-top: 0px; margin-bottom: 0px; }--></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Identifying duplicate depends on what you see as "duplicate".</p>
<p><br>
</p>
<p>I would do the following:</p>
<p><br>
</p>
<p>1. at the beginning of the fix: create a new field "identifier" (for example) that is made by joining other fields</p>
<p>2. use "lookup_in_store" to check if it exists already. </p>
<p>3. If yes, then use "reject", which stops the fix, and rejects your current record<br>
</p>
<p>4. If no, then store the "identifier" using "add_to_store"<br>
</p>
<p>5. do your other stuff<br>
</p>
<p><br>
</p>
<div style="color: rgb(33, 33, 33);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> librecat-dev-bounces@lists.uni-bielefeld.de <librecat-dev-bounces@lists.uni-bielefeld.de> on behalf of Sergio Letuche <code4libuserx@gmail.com><br>
<b>Sent:</b> Friday, December 2, 2016 10:03 AM<br>
<b>To:</b> librecat-dev@lists.uni-bielefeld.de<br>
<b>Subject:</b> [librecat-dev] identify duplicate records with Catmandu</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>Hello community,<br>
<br>
</div>
how do you dedup duplicate records?<br>
<br>
</div>
For a use case we have, we consider duplicate records to be those that share the same content
<br>
<br>
</div>
in for example 245 tag, and all 6** tags.<br>
<br>
</div>
something like a record is identical to another, if in it it has a 245 tag, that has the same value,
<br>
with another record, that has the same metadata in tag 245, or the same metadata in any of the 6** tags.<br>
<br>
</div>
How would you approach this, with a fix?<br>
<br>
</div>
Best <br>
<div>
<div>
<div>
<div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>