How can I define a custom equality operation that will be used by immutable Set comparison methods

equals and hashCode are provided automatically in case class only if you do not define them.

case class MyClass(val name: String) {
  override def equals(o: Any) = o match {
    case that: MyClass => that.name.equalsIgnoreCase(this.name)
    case _ => false
  }
  override def hashCode = name.toUpperCase.hashCode
}

Set(MyClass("xx"), MyClass("XY"), MyClass("xX"))
res1: scala.collection.immutable.Set[MyClass] = Set(MyClass(xx), MyClass(XY))

If what you want is reference equality, still write equals and hashCode, to prevent automatic generation, and call the version from AnyRef

  override def equals(o: Any) = super.equals(o)
  override def hashCode = super.hashCode

With that:

Set(MyClass("x"), MyClass("x"))
res2: scala.collection.immutable.Set[MyClass] = Set(MyClass(x), MyClass(x))

You cannot override the ==(o: Any) from AnyRef, which is sealed and always calls equals. If you tried defining a new (overloaded) ==(m: MyClass), it is not the one that Set calls, so it is useless here and quite dangerous in general.

As for the call to filter, the reason it works is that Set[A] is a Function[A, Boolean]. And yes, equals is used, you will see that function implementation (apply) is a synonymous for contains, and most implementations of Set use == in contains (SortedSet uses the Ordering instead). And == calls equals.

Note: the implementation of my first equals is quick and dirty and probably bad if MyClass is to be subclassed . If so, you should at the very least check type equality (this.getClass == that.getClass) or better define a canEqual method (you may read this blog by Daniel Sobral)