Quick summary of recently reviewed concepts:

| Concept        | Scala 3                    | Rust                          |
|----------------+----------------------------+-------------------------------|
| Type classes   | given/using, derives       | Traits + blanket impls        |
| HKTs           | F[_] natively              | Not directly (GATs partial)   |
| Error handling | Either, Validated, EitherT | Result, ?, thiserror/anyhow   |
| Zero-cost types| opaque type                | newtypes, zero-sized types    |
| Closures       | Function1, SAM types       | Fn/FnMut/FnOnce               |

Other interesting observations:

• cognitive load seems to be much higher when I look at Scala code compared to when I look at Rust’s (tbh I wasn’t expecting this)
• Scala type system expressiveness is huge; I’m not sure I undestand where its depth ends

Scala tooling imporessions

Scala tooling sometimes seem a bit brittle:

1. Code completion sometimes stops working (metals in Emacs/Vim).
2. It is more stable on Intellij in general. Sometimes it has trouble resolving dependencies added to ``build.sbt` though.
3. Intellij sometimes states with its curly underline that an expression is invalid whereas the problem is elsewhere completely (compared to very precise and dev-friendly compiler errors from Rust compiler, it is just not up to my expectations)

Libs impressions

• cats is interesting to read, inspires my imagination for sure, but quite challenging if I want to apply those concepts
• cats-effects and typelevel libs (eg HTTP4s) form an ecosystem that is even harder, something im my head just does not want this complexity
• looking at ZIO and ZIO HTTP: I’m positively surprised with the quality of documentation; yesterday I was able to write a small http server and I had fun :)
• Monix - compared to ZIO - is rather low-level; no Fibers as “first class citizens” (you need to Task.fork), there is CancellableFuture

New resource

I found one resource that is - I think - worth exproring: Software Patterns Lexicon (the link is for Scala, but other languages are there).

The problem

I have the idea of a pipeline: it is defined once, it can be reused and when called, it changes its state but does not evaluate eagerly, because it could be “fed” from an infinite source of data.

For example:

• all integers which are multiple of 13 and has in its binary representation more ones than zeroes
• indices of bytes read from /dev/random which are ascii characters

Perhaps there are two separate issues I’m thinking about here:

1. Is the pipeline creating intermediate collections when processing items through the pipeline? For example, in Scala, all intermediate operations do create a collection (each map and filter)
2. When exactly is the pipeline execution triggered?
3. Can the pipeline computation be reused with different collection? Does it need a collection at all?

In scala

Note: see this insightful comment on SO: https://stackoverflow.com/questions/31664918/does-scala-has-intermediate-terminal-ops-as-java8-has

In the collections library Scala makes a distinction between strict and lazy evaluation. A mechanism for lazy evaluation is called view. By default collections evaluate eagerly at every step.

Let’s create an efectful operation that doubles its input and prints it:

def reportDouble(v: Int): Int = { println(s"Reporting $v -> ${v*2}"); 2 * v }

val l = List(1, 2, 5, 10, 34, 78, 126)

Now, each map or filter on a List evaluates eagerly and produces a list immediately, and the map “traverses” all elements:

l.map(reportDouble).filter(_ > 3).take(3).foreach(println)
# Reporting 1 -> 2
# Reporting 2 -> 4
# Reporting 5 -> 10
# Reporting 10 -> 20
# Reporting 34 -> 68
# Reporting 78 -> 156
# Reporting 126 -> 252
# 4
# 10
# 20

Iterator

We can also create an iterator. Note how the first value gets through the map, even if we haven’t explicitely trigerred it. However, the first element is not actually missing, we will get it eventually when we do next or use other terminal operations causing traversal and evaluation of pipeline operations:

scala>  val it = l.iterator.map(reportDouble).filter(_ > 3)
Reporting 1 -> 2
Reporting 2 -> 4
val it: Iterator[Int] = non-empty iterator

scala> it.next
val res0: Int = 4

scala> it.next
Reporting 5 -> 10
val res1: Int = 10

scala> it.toList
Reporting 10 -> 20
Reporting 34 -> 68
Reporting 78 -> 156
Reporting 126 -> 252
val res2: List[Int] = List(20, 68, 156, 252)

Reusability - Iterator is not reusable

The iterator approach allows to build a pipeline and evaluate elements when needed. However, we are not able to reuse the pipeline: once the iterator is exhausted, we’re done. We need to create a new iterator, repeating the pipeline code. It seems the pipeline needs to have a collection to work on attached to it. Or, in other words, there is no such thing as a pipeline here, I guess.

Being lazy - scala views

views make the pipeline lazy - but this happens only if we have whole pipeline and we don’t store e.g. map result in a val (then - we have no laziness, but eager creation of strict collection). This is the part that was very confusing to me :)

If I create a view and try to map, I am forcing the evaluation! As you can see, all elements of the list get evaluated and printed, and xv hols a SeqView of all evaluated, already-computed values:

# BAD - asiigning l.view.map to val
scala> val xv = l.view.map(reportDouble)
Reporting 1 -> 2
Reporting 2 -> 4
Reporting 5 -> 10
Reporting 10 -> 20
Reporting 34 -> 68
Reporting 78 -> 156
Reporting 126 -> 252
val xv: scala.collection.SeqView[Int] = SeqView(2, 4, 10, 20, 68, 156, 252)

# GOOD - reportDouble does not run again, values are materialized in xv
scala> xv.filter(_ < 20).take(3).foreach(println)
xv.filter(_ < 20).take(3).foreach(println)
Reporting 1 -> 2
2
Reporting 2 -> 4
4
Reporting 5 -> 10
10

Lesson: keep entire chaing on one expression if what I need is the laziness.

View and reusability

Unlike iterator, view is reusable - it holds a reference to the original connection, and it reconstructs the pipeline on each terminal operation, doing a minimal, lazy work on each call (starting fresh from the begining, with minimal processing of what is necessary) - the map is not triggered:

scala> val xv = l.view.map(reportDouble).filter(_ < 20)
Reporting 1 -> 2
Reporting 2 -> 4
Reporting 5 -> 10
Reporting 10 -> 20
Reporting 34 -> 68
Reporting 78 -> 156
Reporting 126 -> 252
val xv: scala.collection.View[Int] = View(2, 4, 10)

scala> xv.take(3).foreach(println)
Reporting 1 -> 2
2
Reporting 2 -> 4
4
Reporting 5 -> 10
10

scala> xv.take(2).foreach(println)
Reporting 1 -> 2
2
Reporting 2 -> 4
4

The answer to reusability is: yes, views can be reused. They still are attached to the source collection- you cannot define a pipeline and apply it to a different collection.

LazyList

Docs: LazyList

LazyLists can be infinite, are lazy and their values are memoized. What does it mean? The pipeline would be evaluated in a way that elements that were previously accessed will not be evaluated again.

The usecase for LazyList are infinite collections which we don’t want to evaluate eagerly (like in the beautiful fibonacci example) and for finite collections don’t give any benefit over views"

LazyList.from(l).map(reportDouble).filter(_ > 3).take(3).foreach(println)
Reporting 1 -> 2
Reporting 2 -> 4
4
Reporting 5 -> 10
10
Reporting 10 -> 20
20

Conclusions for Scala

For lazy, efficient pipelines it is a good idea to use aview, and remember to keep the chain *in one expression".

Reusability of a pipeline across different collections is not possible: the best I can think of is to create a function that takes a collection or an iterator. Similar to Java Streams, we can talk about terminal and intermediate operations, but intermediate does not mean lazy in this case, becasue Scala collections API is by default strict and will evaluate everyting unless you opt out using views (now: think about Rust - it is lazy by default).

At least this is my understanding.

Python

In Python, mapping and filtering operations leave you with lazy iterator objects which by itself do not trigger anything - nothing is evaluated until consumed.

li = [1, 2, 5, 10, 34, 78, 126]

def reportDouble(v):
    print("Reporting {v} -> {v*2}")
    return v * 2

map(reportDouble, li)
# <map at 0x7b77bd1bf5e0> -- no output
filter(lambda v: v < 15, map(reportDouble, li))
# <filter at 0x7b77bc4fd9c0> -- still nothing

Evaluation happens element-by-element only when something consumes those iterators. One needs to explicitely create a list out of the iterators created by map or filter.

fm = filter(lambda v: v < 15, map(reportDouble, li))

[i for i in fm]

# Reporting 1 -> 2
# Reporting 2 -> 4
# Reporting 5 -> 10
# Reporting 10 -> 20 (processed but filtered out)
# Reporting 34 -> 68  (processed but filtered out)
# ...
# result: [2, 4, 10]

Note that all elements get processed - map and filter don’t do any short-circuiting…

I remember that Python has itertools module, perhaps I can find some answers there?

Build-in map and filter suffer from reusability problem: they produce just single-use iterators:

fm = filter(lambda v: v < 15, map(reportDouble, li))

list(fm)  # works, prints all, returns [2, 4, 10]
list(fm)  # returns [] - iterator is exhausted

The Fix - wrap it

If we wrap the source in a function then each time the function is called, new iterator will be created, allowing to reuse the pipeline:

def pipeline(source):
  return filter(lambda v: v < 15, map(reportDouble, source))

list(pipeline(li)) # [2, 4, 10]
list(pipeline(li)) # [2, 4, 10] - works again <- new iterator

If the pipeline is complex (or we want to keep complex state, eg. when traversing a tree) and we want multiple conditions or early exit, or we want it to be lazy then we should use generators, and also itertools.islice(python doc) which is an equivalet of take:

def gen_pipeline(source):
  for v in source:
    doubled = reportDouble(v)
    if doubled < 15:
      yield doubled

# lazy short-cicquit, take only 3 elems
import itertools
list(itertools.islice(gen_pipeline(li), 3))

# Reporting 1 -> 2
# Reporting 2 -> 4
# Reporting 5 -> 10
# result: [2, 4, 10] - stops after 3, does not process rest
# 

Python - summary

Python’s map/filter are lazy but single-use. The language, as far as I know, does not have a “pipeline” construct - the most similar thing I can think of is a function wich can be used to start new iterators when needed - a callable is the reusable abstraction. It cannot be, however, independent from the source: you always need to have a collection or iterator to iterate over.

I once came quite close to the idea of a structure of computations that is independent on the collection - but never actually tried it. It is time to dive into…

Clojure

… clojure’s transducers.

Perhaps the closest concept to what I am searching for is the transducer concept I encountered in clojure. It seems like a definition or composition of operations which do not require a collection and can exist by itself. The documentation states that operations such as map, filter or take (for others see also cheatscheet) create transducers:

What the docs say is:

Most sequence functions included in Clojure have an arity that produces a transducer. This arity omits the input collection; the inputs will be supplied by the process applying the transducer. Note: this reduced arity is not currying or partial application.

Deriving from the documentation example, I can create xf transducer which is a composition of other transducers (comp is composing right-to-left, so the mental model for final transucer operation is that it executes left-to-right, first filtering, then mapping and finally taking two elements). Final reducing operation is “+” here; transduced sequence before addition can be retrieved by using (into [] ...) form:

user> (def xf (comp (filter odd?) (map inc) (take 2))) 
#'user/xf
user> (transduce xf + [3,4,5,6,7,8,9,10,11,12])
10
user> (into [] xf  [3,4,5,6,7,8,9,10,11,12])
[4 6]

What happens when I define xf such that it prints an element, let’s say: very early, before filter? Would It then print everything? At least, it will not at the time transducer is defined. I expect it to fire for all vec elements, however. I aslo expect that if I place print after take, I will print two filterred and inced values, 4 and 6:

Let’s check. Define a: it takes value a, prints it and returns it.

user> (defn  a [a] (println "* " a " *") a)
#'user/a
user> (map a [1 2 50])
*  1  *
*  2  *
*  50  *
(1 2 50)

Transducer xf now is composed of (map a) as first element of the pipeline. Wow, see what happened: it not only did not call map on each element of the sequence - it only executec exectly 3 times - minimal number of times needed to actually retrieve two elements required by last element of the pipeline…:

user> (def xf (comp (map a) (filter odd?) (map inc) (take 2)))
#'user/xf
user> (transduce xf + [3,4,5,6,7,8,9,10,11,12])
*  3  *
*  4  *
*  5  *
10
user> 

Interesting.

In Rust

This behavior is similar to what we get in Rust - there are iterators everywhere :) For example, below program constructs an infinite collection (wait! it is iterator!) and traverses it:

let result: Vec<String> = (0u64..)
    .filter(|n| n % 2 == 0)
    .map(|n| n * n)
    .take_while(|&n| n < 1_000_000)
    .filter(|n| n.to_string().starts_with('4'))
    .map(|n| format!("val:{n}"))
    .collect();
// The entire chain runs element-by-element — no intermediate Vec

So the example with simple collection will be as follows:

fn report_double(v: i32) -> i32 {
    let doubled = v * 2;
    println!("Reporting {} -> {}", v, doubled);
    doubled
}
fn simple_pipeline() -> Vec<i32> {
    let v = vec![1, 2, 5, 10, 34, 78, 126];
    v.iter()
        .map(|n| report_double(*n))
        .filter(|v| *v < 20)
        .take(3)
        .collect()
}
fn main() {
    println!("Example pipeline: {:#?}", simple_pipeline())
}

which behaves nicely - does not go through whole collection and gives exactly what I expected = three elements:

Reporting 1 -> 2
Reporting 2 -> 4
Reporting 5 -> 10
Example pipeline: [
    2,
    4,
    10,
]

Making the pipeline more generic can be done by making collection’s element generic or by creating a function that takes an Iterator<Item=i32> - which allows to use the pipeline on both vec and set (note non-deteministic output):

fn pipeline<T: Iterator<Item = i32>>(it: T) -> Vec<i32> {
    it.map(|n| report_double(n))
        .filter(|v| *v < 20)
        .take(3)
        .collect()
}
fn main() {
    println!("Print pipeline:");
    println!("{:#?}", pipeline(vec![3, 4, 3, 100, 200].into_iter()));
    println!(
        "{:#?}",
        pipeline(HashSet::from([3, 4, 3, 100, 200]).into_iter())
    );
}

or by also returning an iterator - in this case I need to force collection (note a call to .collect::<Vec<i32>>())to be able to print the results as iterator itself does not implement Display:

fn pipeline<T: Iterator<Item = i32>>(it: T) -> impl Iterator<Item = i32> {
    it.map(|n| report_double(n)).filter(|v| *v < 20).take(3)
}
fn main() {
    println!("Print pipeline:");
    println!(
        "{:#?}",
        pipeline(vec![1, 1, 3, 4, 3, 100, 200].into_iter()).collect::<Vec<i32>>()
    );
    println!(
        "{:#?}",
        pipeline(HashSet::from([1, 1, 3, 4, 3, 100, 200]).into_iter()).collect::<Vec<i32>>()
    );
}

which gives following output (obviously, duplicates disappeared on HashSet construction, and set output changes in each execution of the program):

 Print pipeline:
Reporting 1 -> 2
Reporting 1 -> 2
Reporting 3 -> 6
[
    2,
    2,
    6,
]
Reporting 4 -> 8
Reporting 200 -> 400
Reporting 100 -> 200
Reporting 1 -> 2
Reporting 3 -> 6
[
    8,
    2,
    6,
]

Conclusions

Let’s go back to initial question: can I define a pipeline once, keep it lazy and reuse with different inputs?

This question has two separate parts: laziness and reusability, and different languages answer them in a different way.

Python is lazy: map and filter return iterators which are evaluated lazily element-by-element when they are consumed (in terminal operations, like creating a list()), and those iterators are single use: when exhaused, cannot be re-run. To re-use, you can define a function; to short-cirquit, you can use itertools.islice.

Scala defaults to strict evaluation - every map and filter on a List creates a full intermediate collection. You can opt-out using .view which defers all operations until a terminal foreach which will process what’s needed. view-based pipeline can be reused: it holds reference to original collection, and rebuilds the lazy chain on each terminal call - but it needs to be a single expression! (Confusing part: if you assign intermedite result to a val it will force evaluation at that point and break laziness.)

Rust makes laziness default - iterators cause evaluation elem-by-elem, there is no intermediate collections created, and those pipelines short-cirquit nicely with take or take while. If you define a pipeline inside a function that takes impl Iterator, it is (somewhat) independent from the input: you can pass vec, set, infinite range. Very elegant.

Clojure has a concept of a transducer - it is a pipeline definition that is completely decoupled from the source and the collection of results: you define complex transducers by combining simpler transducers using comp and then uou can appply it to different sinks (transduce, into) It evaluates lazily and minimally - only as many elements are processed as needed. Of all four langiuages I experimender with, transducers in Clojure are closest to my idea of an independent pipeline.

Depending on what you need, you can choose what you want! What I needed for sure was the above journey - it let me to order the concept of pipelined computations in my head. Jumping bewteen languages and using all of them at the same time sometimes casues the concepts leak between areas of my brain, causing a bit of confusion :D

Happy hacking!

1. Exercism Completed Phase 1 week 3 exercises on exercism:
   1. hello-world
   2. two-fer
   3. raindrops
   4. bob
   5. hamming
   6. pangram
   7. isogram
   8. difference-of-squares

2. Played with function composition in scala - implemented simple banner generator (see header of this and previous entry).

Also, learned about external process calling

  object Colors {

    abstract class ColorSource {
      //generate colors from string once
      lazy val into: List[Color] = {
        val hx = hexes()
        println(hx)
        hx.map(Color.decode)
      }
      def hexes(): List[String]
      def random: Color = into(r.nextInt(into.length))
    }

    class Pastel extends ColorSource {
      def hexes(): List[String] = {
        import scala.sys.process._
        val result = (Seq("pastel", "random","-s",  "vivid") #| Seq("pastel", "format", "hex")).!!
        result.split("\n").toList
      }
    }

    class Gradient(from: String="ffffcc", to: String="fd8d3c", n: Int = 5) extends ColorSource {
      def hexes(): List[String] = {
        import scala.sys.process._
        val cmd = Seq("pastel", "gradient", "--colorspace", "HSL", from, to, "-n", n.toString) #| Seq("pastel", "format", "hex")
        println(cmd)
        val result = cmd.!!
        result.split("\n").toList
      }
    }

    class Fixed(cols: List[String]) extends ColorSource {
      def hexes(): List[String] = cols              
    }

    def pastel: ColorSource = new Pastel
    def gradient(from: String, to: String, n: Int) = new Gradient(from, to, n)
    def fixed(cols: List[String]) = new Fixed(cols)
	
	/** ... */
}

3. Setting up rustlings failed. I probably messed up the configuration (or had old version of rustlings tool). Worked after removal of whole repo (minus execrises done so far),tool reinstall (cargo install rustlings), rustlings init and copying over my exercises.

4. Learned about traits from https://doc.rust-lang.org/book/ch10-02-traits.html

Problem

I have a directory where I store files: articles, images, notes - in a bit unordered and random fashion: I copy them from other devices, generate some files requried for teaching sessions or create subdirectrories for experimentation.

As a result, this directory is messy and contains a lot of duplicated files. Some of them are quite big (think: .epub or .pdf, not linux distro .iso).

So, I thought, this is a good candiate for a practical problem solving exercise joined with language learning practice for Rust and Scala.

The idea is simple: I will scan the directory recursively and for each file I will calculate the SHA-256 sum and put the sum and path to a hash map, printing those sums for which there is more than one path entry.

Experiment

I decided to write this in scala and in rust.

Results

In Scala I use following case class representing a path with its sha sum:

case class ShaRes(val sha: String, p: os.Path)

Scala with external process called per file

private def usingExternal(p: os.Path): Option[ShaRes] = { 
    val res: os.CommandResult = os.proc("shasum", "-a", "256", p.toString).call()
    List(res.out.text().split("\\s+") *) match {
      case md :: rest => Some(ShaRes(md, p))
      case v => {
        println(Cons.err(s"Could not parse $p"))
        None
      }
    }
  }

I build scala with: scala-cli --power package Main.scala -o books and run it with time ./books /home/karma/Dokumenty/books/

Scala with external process call for each file:

real    0m36,687s
user    0m35,232s
sys     0m4,635s

Scala using MessageDigest:

Here the path is passed to MessageDigets’ update method:

private def usingDigest(p: os.Path): Option[ShaRes] = {
    import java.nio.file.Files
    import java.security.MessageDigest
    
	val md = MessageDigest.getInstance("SHA-256")
    md.update(Files.readAllBytes(p.wrapped))
    val sha = md.digest().map(b => f"$b%02x").mkString
    Some(ShaRes(sha, p))
  }

Result:

real    0m5,642s
user    0m5,899s
sys     0m1,136s

Rust

Sha calculation uses sha2 crate:

fn sha_read<T: Read>(mut r: T) -> io::Result<String> {
    let mut sha256 = Sha256::new();
    io::copy(&mut r, &mut sha256)?;
    let hash = sha256.finalize();
    let result = format!("{:x}", hash);
    Ok(result.to_string())
}
pub fn sha(fname: &str) -> io::Result<String> {
    let file = File::open(fname)?;
    sha_read(file)
}

and iterates over directory entries using walkdir crate. ShaStore datastructure aims to mimic hashmap approach from scala:

#[derive(Debug)]
struct ShaStore<'a> {
    shas: HashMap<String, Vec<&'a Path>>,
    errors: Vec<String>,
}
impl<'a> ShaStore<'a> {
    fn new() -> Self {
        ShaStore {
            shas: HashMap::new(),
            errors: Vec::new(),
        }
    }
    fn update(&mut self, pref: &'a Path) -> io::Result<()> {
        let shares = pref.to_str().map(|p| sha(p));
        match shares {
            Some(Ok(shaval)) => {
                if let Some(v) = self.shas.get_mut(&shaval) {
                    v.push(pref);
                } else {
                    self.shas.insert(shaval.clone(), vec![pref]);
                }
            }
            Some(Err(e)) => {
                self.errors.push(e.to_string());
            }
            None => self.errors.push("No sha".to_string()),
        };

        Ok(())
    }
}

Rust with calculation:

real    0m4,796s
user    0m3,846s
sys     0m0,704s

First conclusion

Calling a subprocess and wait synchronously is a huge overkill. It seems that digest calculation in-process is really very fast - the difference between scala and rust is not very significant. Well, most of the time we’re just accessing filesystem and calculating sha.

• individual sha calculations are independent and can be done in parallel
• what datastructure in Scala is safe for concurrent acces?
• in scala I can try to spawn a Future; in Rust I could Well, perhaps it is a good moment to try and introuduce some concurrency?

Option types

Option type documentation:

• Scala Option
• Rust Option

Same names, a bit different API

Scala

In Scala the variants are called: None and Some which are final case subclasses of Option. I’ve noticed the subclassing has the nice property of helping in IDE to discover the variants (which is also the case, for example, for Try subclasses.

Rust

In Rust, the variants are also conveniently named Some and None.

API differences

(Scala vs Rust)

• check presence: isDefined vs is_some
• check absence: isEmpty vs is_none
• transform value: map vs map
• chain options: flatMap vs and_then
• filter value: filter vs filter
• default value: getOrElse vs unwrap_or
• lazy default: getOrElse vs unwrap_or_else
• alternative: orElse: vs or
• to collection: toList vs iter
• unsafe extract: get vs unwrap

Some specifics

• Scala has some functions that
  • does not make sense in Rust but has when interop with Java is required (orNull - Rust has no null by design)
  • convert to Either subtypes (toLeft, toRight)
• Rust has functions
  • to convert between reference types (as_ref, as_mut, as_deref)
  • to modify in place or work with ownership (take, replace)
  • to transform to Result (ok_or, transpose)

Exercise 1 - compare Rust and Scala

This is the example from Rust module option docs:

enum Kingdom { Plant(u32, &'static str), Animal(u32, &'static str) }

// A list of data to search through.
let all_the_big_things = [
    Kingdom::Plant(250, "redwood"),
    Kingdom::Plant(230, "noble fir"),
    Kingdom::Plant(229, "sugar pine"),
    Kingdom::Animal(25, "blue whale"),
    Kingdom::Animal(19, "fin whale"),
    Kingdom::Animal(15, "north pacific right whale"),
];

// We're going to search for the name of the biggest animal,
// but to start with we've just got `None`.
let mut name_of_biggest_animal = None;
let mut size_of_biggest_animal = 0;
for big_thing in &all_the_big_things {
    match *big_thing {
        Kingdom::Animal(size, name) if size > size_of_biggest_animal => {
            // Now we've found the name of some big animal
            size_of_biggest_animal = size;
            name_of_biggest_animal = Some(name);
        }
        Kingdom::Animal(..) | Kingdom::Plant(..) => ()
    }
}

match name_of_biggest_animal {
    Some(name) => println!("the biggest animal is {name}"),
    None => println!("there are no animals :("),
}

Let’s see how it would look like in scala:

  private object Kingdom {
    final case class Plant(age: Int, name: String)
    final case class Animal(age: Int, name: String)
  }

  private val allTheBigThings = Vector(
    Kingdom.Plant(250, "redwood"),
    Kingdom.Plant(230, "noble fir"),
    Kingdom.Plant(229, "sugar pine"),
    Kingdom.Animal(25, "blue whale"),
    Kingdom.Animal(19, "fin whale"),
    Kingdom.Animal(15, "north pacific right whale"),
  )

  private def findBiggest(): Unit =  {
    var nameOfTheBiggestAnimal: Option[String] = None
    var sizeOfTheBiggestAnimal = 0
    allTheBigThings.foreach {
      case Kingdom.Animal(age, name) if age > sizeOfTheBiggestAnimal =>
        sizeOfTheBiggestAnimal = age
        nameOfTheBiggestAnimal = Some(name)
      case _ => ()
      // or:
      // case Kingdom.Animal(_,_) | Kingdom.Plant(_,_) => ()

    }

    nameOfTheBiggestAnimal match {
      case Some(name) => println(s"the bigest animal is $name")
      case None => println("There are no animals :(")
    }
  }

My notes:

• in scala 2.x (before enum and named tuples) modelling two similar variants is possible either by
  • overriding abstract class Kingdom in case subclasses (would preserve class hierarchy) or
  • scoping case class Plant and case class Animal in object Kingdom which allows, well, scope access (which I like very much - this approach pushes towards providing explicit path to a class)
• forEach allows to pattern match in anonymous function - without defining any binding to a iteration variable (big_thing in Rust snippet)
• scala requires to declare a type of nameOfTheBiggestAnimal - without it the code does not compile; that’s because None value resolves to None.type and Some[String] is expected in assignment: compiler cries until I fixed that:

Exercise 2 - rewrite with map/flatmap

Readability matters. If we have this simple model:

case class User(name: String, email: Option[String])
case class Prefs(theme: Option[String])

val users = Map(1 -> User("Alice", Some("a@b.com")), 2 -> User("Bob", None))
val prefs = Map("a@b.com" -> Prefs(Some("dark")))

then we can use (preferred! 😊) scala for:

def getTheme(userId: Int): Option[String] =
  for
    user  <- users.get(userId)
    email <- user.email
    pref  <- prefs.get(email)
    theme <- pref.theme
  yield theme

without which we would have to do this (don’t! 😤):

 def getTheme2(userId: Int): Option[String] = {
  users
	.get(userId)
    .flatMap(u => u.email)
	.flatMap(e => prefs.get(e))
	.flatMap(p =>p.theme)
}

Note

Image was generated with this scala script:

That’s all for today.

Happy coding!

Science based programming language learning

Exploring two prorgamming languages at the same time seems like a bad idea, doesn’t it? Well, good learning sources say that the best approaches to effective, expert learnings are:

• interleaving (study both languages concurrently — never sequentially)
• spaced repetition
• deliberate practice at the edge of your comfort zone
• retrieval practice (close the docs)
• and the explanation effect

Looks like a very good idea to me.

• What if I could somehow optimize the learning process by utilizing all of them?
• What if I create a protocol with stuff I want to learn split into some stages and implement those science-based approaches to both “ingest” and “practice”?

Risks:

• mental overwhelm - burnout, self-blame and all the internal struggle
• ADD effect of not sticking to the protocol

So I engaged AI for preparing

• the plan for learning Scala and Rust in parallel
• the Anki deck for learning
• and the SPA with the deck as well

The plan

This is the suplementary plan to the one I generated specifically for Scala in Scala Learning Path, however:

• original plan covers Scala only and is defined for 8 weeks
• this plan covers both languages, Scala AND Rust

Resources

• 🤩 Curriculum: (online page): Online Curriculum (<- explore the tabs!)

• 📗 Deck for Anki learning (download link): Anki Deck

• 🗓️ Org-agenda (for Emacs users - download link): Org-agenda file

I think I’m ready!

The execution

As an exercise, after I wrote simple anniversary lister in Rust (see diary) I created the same program in Scala (see scala-diary).

What I have learned

1. Once you are slighly more familiar with one language your tendency is to use it for all auxiliatory tools. For example, to generate a list of events as input for both Rust and Scala versions I reach out to Rust (see Diary Generator)
2. Writing unit tests inside a source file and not in a different directory helps to practice DDD, eliminating the need to switch between production and test code
3. Modelling errors: I used enums - in Rust I’m aware of crates like thiserror and anyhow (two top-level on rates.io in error-handling category)
4. Enums: in Scala I used Scala Enums as Left of the Either; in Rust I also modelled them using Rust Enums in Result type.

• those are different. Scala enums:
  • are “used to define a type consisting of a set of named values” (in scala 2 they were represented as objects)
  • are based on reflection
  • can be used as Java Enums by extending java.lang.Enum:
  enum ScalaEnum extebds Enum[ScalaEnum]:
    case ValueOne, ValueTwo

  • can have fields - and different shapes:
    enum EBase(val n:Int):
      case EOne(val s: String) extends EBase(1)
      case ETwo(val d: java.time.Duration) extends EBase(2)

  but then you cannot use EnumClass.fromOrdinal (obviously) which works only for non-parameterized enums (which still can have some constant data):
  enum Foo(val n: Int):
     case Bar extends Foo(7)
     case  Baz extends Foo(345)
  
  scala> Foo.Bar.n
  val res12: Int = 7
  
  scala> Foo.fromOrdinal(0)
  val res15: Foo = Bar
  
  scala> Foo.fromOrdinal(0).n
  val res16: Int = 7

  • previously (in scala 2.x) the pattern matching with exhaustiveness checking and with possibility to use different data required reaching out for sealed class hierarchies:
     sealed class Base(val n: Int)
     case class One(s: String, t: java.time.LocalDate) extends Base(1)
     case class Two(d: java.time.Duration) extends Base(2)

  To be honest, the only valuable thing enums bring to the table - afaik - is its Enum’s api (.values, fromOrdinal(Int)) - as stated in this explanation - enums is just a syntactic sugar 🍭 for sealed classes.
• Rust Enums are quite similar I think:
  • variants can have different shapes
  • behave more like a sealed class hierarchy in Scala
  • can be pattern-matched as well

5. It is so cool to have scala REPL especially at the beginning of the journey. Rust does not have that.

Resources

For reading

• Rust: it is cool to have
  • free Rust Book
  • there is also Cargo Book
• Scala:
  • official docs
  • articles on RockTheJVM blog
  • wip sbt book 2.x
  • sbt book 1.x

For experimenting

• Rust: https://play.rust-lang.org/?version=stable&mode=debug&edition=2024
• Scala: https://scastie.scala-lang.org/ (or scala-cli console once you’re done with installing coursier - which, to be honest, I don’t fully grok yet)

Happy Hacking!

Perhaps Gemini could plan my learning of Scala?…

Yes, it can.

See:

• My learning path as .pdf,
• My learning path as .typ sources,

I wish me good luck.

Credits

Gemini.

Discovery

Leaving a familiar ground of Java, Maven and Spring Boot, and switching back to another JVM language - Scala - shouldn’t be a Big Deal, right?

Especially when once upon a time I did complete two Cursera courses about Scala and Functional Programming, which - as far as I remember - were really eye-opening, rewarding and funny.

Let’s dive into remote memories…

When I was learning Clojure, I had one issue (no, it was not about parenthesis; those serve me well) and it was: learning Emacs. An adventure of its own. Also, some language concepts needed to settle in my head. My fingers needed to learn paredit shortcuts. And I had a lot of fun.

Kotlin was even more fun, and that’s because it has brought a feeling of a deep relief: similar to Scala, but much simpler; with familiar non-lispy syntax; better Java and without such scary typesystem (not to mention I could use an excellent Jetbrains IDE).

I played with Golang and Rust. Both languages have a valuable property of being sufficienlty “cathedral-ic” to have:

• an excellent commandline tooling
• comprehensive webpage
• a comitee/team which decides the direction the language would follow
• top-noth package / build management system (I ❤️Cargo!)
• crazy smart devs doing briliant software (and a lot of it!)

Now, I decided to refresh my memory and check how it is to learn and program with Scala. So I left my Java land and set my navigation tools in the direction of Scala.

Confusion

First, I needed to install something, probably a scala compiler.

Which one

Wait, there are two Scalas:

• Scala 2 (which seem not to be binary compatible between releases)
• and Scala 3 (which follows its own compatibility rules

Oh, do we have Python2 vs Python3 story here? Oh, and each has its own LTS?

Good. It should not matter which Scala I use if I have a good tooling.

What should I install

I should follow scala-lang instructions, so I installed coursier.

• It installed scala-cli - not sure why I need it if for dependencies I have coursier and sbt, and for scripting I have ammonite? Ah, it has using directives, runs .sc files which might have top level statements
• it installed sbt which looks a bit complex to grasp (see sbt docs) - my mental model is that it is maven replacement in Scala ecosystem
• also, ammonite - which should helo with scala scripting - worth checking out (udpate: it breaks on Ubuntu)
• and scalafmt - nice, I hope scala code can be well formatted automatically!

Do I need all of it? I’m confused.

Metals

There is a language server metals which just does not want to play well with VisualStudioCode. It just stops working. I’m confused again.

Interestingly, it kinda works when I use helix editor - it is explicitly supported but LSP features don’t work as expected (compared to eg LSP support for Rust)

It supports Emacs, but after looking at the emacs config example I’m not sure I want to use it.

Intellij

In the end, I switched to Intelij as it seems to be more stable than Metals with VSCode. However, Scala plugin is it saying strange things to me sometimes and also shows red curly lines and then those lines disappear suddenly. So I’m a bit confused.

Hands to code

I found scala toolkit and played a bit with it. Nice. On the toolkit introduction page I found six ways of running the code. I looked at mill, tried it and confused some commands with sbt.

If mill is simpler than sbt, then fine, but I don’t do sbt-complex things (yet) so I’ll stick with sbt. Unless I get more confused with sbt than I am with mill.

Libraries

• cats
• cats effects
• tapir

Would I live long enough to get it? I need a good Algebra book first I guess.

Today I learned on youtube that “effects” in “cats effects” have nothing to do with “side effects”. Confusion, again.

The journey begins

Even the most daunting and massive undertakings are accomplished through a series of small, manageable actions.

“A journey of a thousand miles begins with a single step.”

Lao Tzu in “Tao Te Ching”

Perhaps Gemini could plan my learning of Scala?…

Credits

Happy cats and tapirs in server room generated with Gemini.