I agree with R. Pate and Todd Gardner; a std::set might be a good idea here. Even if you’re stuck using vectors, if you have enough duplicates, you might be better off creating a set to do the dirty work. Let’s compare three approaches: Just using vector, sort + unique sort( vec.begin(), vec.end() ); vec.erase( … Read more
array = array.uniq uniq removes all duplicate elements and retains all unique elements in the array. This is one of many beauties of the Ruby language.
I would suggest using the duplicated method on the Pandas Index itself: df3 = df3[~df3.index.duplicated(keep=’first’)] While all the other methods work, .drop_duplicates is by far the least performant for the provided example. Furthermore, while the groupby method is only slightly less performant, I find the duplicated method to be more readable. Using the sample data … Read more
A really easy way to do this is to add a UNIQUE index on the 3 columns. When you write the ALTER statement, include the IGNORE keyword. Like so: ALTER IGNORE TABLE jobs ADD UNIQUE INDEX idx_name (site_id, title, company); This will drop all the duplicate rows. As an added benefit, future INSERTs that are … Read more
Editor warning: This solution is computationally inefficient and may bring down your connection for a large table. NB – You need to do this first on a test copy of your table! When I did it, I found that unless I also included AND n1.id <> n2.id, it deleted every row in the table. If … Read more
SELECT DISTINCT a,b,c FROM t is roughly equivalent to: SELECT a,b,c FROM t GROUP BY a,b,c It’s a good idea to get used to the GROUP BY syntax, as it’s more powerful. For your query, I’d do it like this: UPDATE sales SET status=”ACTIVE” WHERE id IN ( SELECT id FROM sales S INNER JOIN … Read more
I like CTEs and ROW_NUMBER as the two combined allow us to see which rows are deleted (or updated), therefore just change the DELETE FROM CTE… to SELECT * FROM CTE: WITH CTE AS( SELECT [col1], [col2], [col3], [col4], [col5], [col6], [col7], RN = ROW_NUMBER()OVER(PARTITION BY col1 ORDER BY col1) FROM dbo.Table1 ) DELETE FROM … Read more
If you don’t want duplicates in a Collection, you should consider why you’re using a Collection that allows duplicates. The easiest way to remove repeated elements is to add the contents to a Set (which will not allow duplicates) and then add the Set back to the ArrayList: Set<String> set = new HashSet<>(yourList); yourList.clear(); yourList.addAll(set); … Read more
If you’re using .Net 3+, you can use Linq. List<T> withDupes = LoadSomeData(); List<T> noDupes = withDupes.Distinct().ToList();
To remove duplicates use set(a). To print duplicates, something like: a = [1,2,3,2,1,5,6,5,5,5] import collections print([item for item, count in collections.Counter(a).items() if count > 1]) ## [1, 2, 5] Note that Counter is not particularly efficient (timings) and probably overkill here. set will perform better. This code computes a list of unique elements in the … Read more