In my previous job, we were working on backend system that regularly fetched data from other services, both 3rd party and inhouse. These data were parsed and mapped into our internal data structure modelled with case classes. The fetching itself was usually done using some REST API and was mostly straightforward, except for one service, where we had to explicitly list all fields of given entity for which we wanted to fetch new data. This was done using REST call similar to this:

GET /entities/User?fields=id,name,surname

So based on the fact that this entity would be mapped in our backend as case class User, how can we get all its field names to build the above query? In this blog post, I’d like to share the simple solution, based on some generic derivation using Shapeless.

1 Before we start

This blog post assumes at least basic knowledge of what Shapeless is and what is the HList and the Aux pattern. Here is the list of some resources to start with:

2 Naive implementation

Let’s start with the definition of the example case class:

case class User(id: Long, name: String, surname: String)

So the goal is clear, we need to write piece of code that extracts field names from any given case class, without the need to reimplement it again and again for every case class in our domain. So instead of working with concrete types (let’s say the User case class), we want to work with more generic representation, something like list of fields.

2.1 Attempt with Generic

Let’s try to find way how to convert any case class into its generic representation. Shapeless offers Generic class, that does this for us:

import shapeless._

val genUser = Generic[User]
// genUser: Generic[User]{type Repr = Long :: String :: String :: shapeless.HNil}

As you can see, the Repr type represents the generic representation of our case class, encoded as HList. But problem with this representation is that it only contains field types, not the names. To solve this, we need to use different class, the LabelledGeneric.

2.2 LabelledGeneric to the rescue

LabelledGeneric is similar to Generic, but it also encodes field names:

import shapeless._

val genUser = LabelledGeneric[User] 
genUser: LabelledGeneric[User]{type Repr = Long with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("id")],Long] :: String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("name")],String] :: String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("surname")],String] :: shapeless.HNil}

Ok, the Repr type now looks pretty complex, but you can see the pattern there: each field is represented by its type with KeyTag, which contains the field name. The question is, how to extract these field names to something like List("id", "name", "surname")? This is exactly what the Keys class is for.

2.3 Extracting field names

import shapeless._
import shapeless.ops.record._

val genUser = LabelledGeneric[User] 
val keys = Keys[genUser.Repr].apply 
// keys: Symbol with tag.Tagged[id] :: Symbol with tag.Tagged[name] :: Symbol with tag.Tagged[surname] :: HNil = 'id :: 'name :: 'surname :: HNil
val keyList = keys.toList.map(_.name)
// keyList: List[String] = List("id", "name", "surname")

Et voilà. We got the list of field names, exactly as needed. But problem is that this code is specific to the User case class. So let’s do the final step and make the code more generic.

3 Making things more generic

Here is the code for final, generic solution:

trait Attributes[T] {
  def fieldNames: List[String]
}

object Attributes {
  implicit def toAttributes[T, Repr <: HList, KeysRepr <: HList](
    implicit gen: LabelledGeneric.Aux[T, Repr],
    keys: Keys.Aux[Repr, KeysRepr],
    traversable: ToTraversable.Aux[KeysRepr, List, Symbol]
  ): Attributes[T] = new Attributes[T] {
    override def fieldNames: List[String] = keys().toList.map(_.name)
  }

  def apply[T](implicit attributes: Attributes[T]): Attributes[T] = attributes
}

As you can see, it’s pretty similar to previous code, but with few differences:

  • we need to use the Aux pattern now to overcome some limitations of Scala’s type system
  • we need to obtain implicit value for ToTraversable, because it’s required by the following call keys().toList.map(_.name)

Also instances of Keys, LabelledGeneric, etc. are now resolved from implicit values. These are basically provided by shapeless, backed up by some heavy macro machinery, but we really don’t need to care about this detail. Rest of the code is pretty much same, just wrapped up into the Attributes trait, so we can use it like this:

val keys = Attributes[User].fieldNames
// keys = List(id, name, surname)

4 Scala 2.13 and productElementNames

Scala 2.13 added new method, productElementNames, into the Product trait, from which all case classes inherits. This method returns list of all field names, but since it’s method, it needs existing instance, which is actually the major drawback, compared to the above solution:

val user = User(123L, "John", "Smith")
val keys = user.productElementNames.toList
// keys: List[String] = List(id, name, surname) 

Because of this fact, it really cannot be used as replacement of the Shapeless based implementation, but I decided to mention it here just for the sake of completeness.