Category: Reference


Lossy decodable for arrays

August 3, 2019

General Coding, Reference

Comments Off on Lossy decodable for arrays


This article is about exploring a way to decode only the “good” items from arrays.

It’s common for apps to decode arrays of data. For example, you may have a feed of user-generated content or a list of items for sale. To get this data, the app will make a network request to some backend API. Then, that API will most likely send the data back as JSON.

Swift gives us a great way to decode such data. You can simply set your objects to conform to the Decodable protocol. Then use JSONDecoder to build your objects.

Unfortunately, the data isn’t always perfect. If the data in the JSON doesn’t match your model the decoder will throw an error. And if just one field of an object or sub-object isn’t right, the entire list is thrown out.

So, what can we do about it? How can we allow the good items to go through and only reject the bad items without rejecting the entire list?

Starting with good data

Let’s look at an example. We’re going to decode a page of messages.

struct Message: Decodable {
    let sender: String
    let subject: String?
    let body: String
}

struct MessagePage: Decodable {
    let page: Int
    let limit: Int
    let items: [Message]
}

We have a page of message items which each contain a sender, optional subject and body.

Lets look at some good test data:

let goodTestData = """
{
    "page": 1,
    "limit": 10,
    "items": [
        {
            "sender": "Sender One",
            "subject": null,
            "body": "Body one."
        },
        {
            "sender": "Sender Two",
            "body": "Body two."
        },
        {
            "sender": "Sender Three",
            "subject": "Third subject",
            "body": "Body three."
        }
    ]
}
""".data(using: .utf8)!

Since the subject is optional, it can be null, missing, or contain a valid value and the decoder will easily handle each of those cases.

Finally, we can test decoding this data with something like this:

let jsonDecoder: JSONDecoder = {
    let decoder = JSONDecoder()
    decoder.keyDecodingStrategy = .convertFromSnakeCase
    return decoder
}()

do {
    let posts = try jsonDecoder.decode(MessagePage.self, from: testData)
    posts.items.forEach { print($0) }
} catch {
    print(error)
}

So far, there’s no issue. We tested the expected good data and everything is properly decoded. Ship it! Unfortunately, the real world is full of poorly constructed data. Our assumptions may be false.

Dealing with bad data

What if we were working with this data instead:

let testData = """
{
    "page": 1,
    "limit": 10,
    "items": [
        {
            "sender": "Sender One",
            "subject": null,
            "body": "Body one."
        },
        {
            "sender": "Sender Two",
            "body": "Body two."
        },
        {
            "subject": "Third subject",
            "body": "Body three."
        }
    ]
}
""".data(using: .utf8)!

Now we don’t get any items. Instead, we get an error telling us that our third item is missing a value for “sender”.

We could go back to our model and make sender optional. In some cases, it might make sense, but what does it mean if a message has no sender? This data may be required by our UI or even for another API that the app is using. Besides, if we go make all the fields optional we might as well just have used the older JSONSerialization instead.

One option would be to use manually implement the initializer and skip over any bad items by decoding into a dummy object.

Custom Decodable

struct Dummy: Decodable { }

struct MessagePage: Decodable {
    let page: Int
    let limit: Int
    let items: [Message]
    
    enum CodingKeys: CodingKey {
        case page, limit, items
    }
    
    init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        page = try container.decode(Int.self, forKey: .page)
        limit = try container.decode(Int.self, forKey: .limit)
        var items = [Message]()
        var itemsContainer = try container.nestedUnkeyedContainer(forKey: .items)
        while !itemsContainer.isAtEnd {
            do {
                let item = try itemsContainer.decode(Message.self)
                items.append(item)
            } catch {
                _ = try? itemsContainer.decode(Dummy.self)
            }
        }
        self.items = items
    }
}

Now we get the good items in the array and allow the bad items to drop off.

We need the dummy object because the index of the decoder doesn’t increment when a decode fails. We can abstract that away with a failable decodable object.

Failable Decodable

struct FailableDecodable<Element: Decodable>: Decodable {
    var element: Element?
    init(from decoder: Decoder) throws {
        let container = try decoder.singleValueContainer()
        element = try? container.decode(Element.self)
    }
}

This allows us to remove the Dummy object and rewrite the initializer as follows:

init(from decoder: Decoder) throws {
        let container = try decoder.container(keyedBy: CodingKeys.self)
        page = try container.decode(Int.self, forKey: .page)
        limit = try container.decode(Int.self, forKey: .limit)
        var items = [Message]()
        var itemsContainer = try container.nestedUnkeyedContainer(forKey: .items)
        while !itemsContainer.isAtEnd {
            if let item = try itemsContainer.decode(FailableDecodable<Message>.self).element {
                items.append(item)
            }
        }
        self.items = items
    }

This isn’t scalable yet, it’s a lot of boilerplate to write anytime you have an array that could fail. Let’s fix that.

Lossy Decodable Array

struct LossyDecodableArray<Element: Decodable>: Decodable {
    let elements: [Element]

    init(from decoder: Decoder) throws {
        var elements = [Element?]()
        var container = try decoder.unkeyedContainer()
        while !container.isAtEnd {
            let item = try container.decode(FailableDecodable<Element>.self).element
            elements.append(item)
        }
        self.elements = elements.compactMap { $0 }
    }
}

Now we can greatly simplify the MessagePage object.

struct MessagePage: Decodable {
    let page: Int
    let limit: Int
    let items: LossyDecodableArray<Message>
}

Looking pretty good. The thing I don’t like is that you have to access the messages using items.elements. So let’s fix it.

extension LossyDecodableArray: RandomAccessCollection {
    var startIndex: Int { return elements.startIndex }
    var endIndex: Int { return elements.endIndex }
    
    subscript(_ index: Int) -> Element {
        return elements[index]
    }
}

Now we can access the elements as we did originally and everything is working nicely.

Conclusion

I like using Decodable models to represent objects returned by backend APIs. Unfortunately, the data isn’t always perfect, and it can be challenging to find a clean solution using Decodable. However, with some persistence and the right abstractions, we can create scalable solutions. I really like how this case turned out, and I hope you do too.

I’d love to hear your feedback, questions or thoughts; find me on twitter @kenboreham

🔥 Thanks for reading! 👍


Add some color to your tvOS buttons

September 30, 2018

Button, Reference, tvOS, UIKit

Comments Off on Add some color to your tvOS buttons


The interface for UIKit is the same between iOS and tvOS, but the behavior is a little different. The interaction on tvOS is distinctly different than iOS because tvOS doesn’t offer direct interaction. Instead of tapping the screen you use a remote. This is an important point when deciding on how to customize interactive elements.

In this article, I’m going to discuss how you can add color to your buttons on tvOS. This is often required for consistency and branding purposes.

There are a couple of options depending on what your requirements are. The problem is that you might have to try a few approaches to get the desired look.

I will quickly go through what I’ve tried, what worked and what didn’t.

Read More »


Master map, compactMap, flatMap, reduce and filter by creating your own implementation

February 21, 2018

General Coding, Playground, Reference

Comments Off on Master map, compactMap, flatMap, reduce and filter by creating your own implementation


There are a few higher-order functions for collections that can be rather difficult to fully understand. While there are several good resources out there to learn the basics, it can still be confusing and seem a bit like magic. And when things seem like magic they tend to not get used effectively or maybe not at all.

In this post, I’m going to try explaining map, compactMap, flatMap, reduce and filter by walking through their inputs, outputs and deriving an implementation for each. For simplicity, we will focus on arrays but this can be expanded to most data structures. The goal is only learning the ideas, not to implement the best possible solution.Read More »


Beyond 128-bit integers

December 24, 2017

General Coding, Reference

Comments Off on Beyond 128-bit integers


I just finished a post related to the top range of integer values that are supported by Swift.

https://kenb.us/big-integers

The conclusion was that we can use the Decimal type to hold integer values with up to nearly 128 bits. That’s an impressively large number.

It’s hard to imagine a case where you would require both a larger number and precision at the least significant figures. However, it does make an interesting exercise so let’s give it a shot.

Read More »


How to store large integer values without losing precision

December 23, 2017

General Coding, Reference

Comments Off on How to store large integer values without losing precision


I recently wanted to use some very large numbers in Swift and quickly came across some limitations.

 

Int

My first implementation started out naively with Int (Int64) type. This is, after all, the easiest way to use integers.

The limitations are also easy to understand.
Read More »


3 ways to extract numbers from a string

December 3, 2017

General Coding, Performance, Reference

Comments Off on 3 ways to extract numbers from a string


I’m working on a project which requires me to parse some data that was formatted more for readability than parsing. Similar to NSStringFromCGRect which outputs in the format “{{x,y},{width,height}}”

let rect = CGRect(x: 124, y: 387, width: 74, height: 74)
let text = NSStringFromCGRect(rect)
// "{{124, 387}, {74, 74}}"

This string can easily be read back again using it’s counterpart function NSStringFromRect.

let text = "{{124.123, 387}, {74, 74}}"
let rect = NSRectFromString(text)

Read More »


How to get UITextView to behave like UILabel

November 20, 2017

Reference, Text, UIKit

Comments Off on How to get UITextView to behave like UILabel


UILabel and UITextView both rely on TextKit to render text. So why do they behave differently and how can they work together?

Below is a view with a UILabel (blue) and two UITextViews (red) laid out in IB.

The Label is constrained to the top right corner of the view and uses it’s intrinsic content size to determine the width and height. The size of the label matches closely to the size of the text.

Read More »

Starting a new iOS project

November 19, 2017

Reference

Comments Off on Starting a new iOS project


Most of this is going to be rather straight forward. Most developers will have done this plenty of times. So why bother writing about it? In my case, there are several things that I tend to do for most projects. I find that, if don’t do these things up front, I’ll usually end up in a place that I’ll need to do it at that time. These steps don’t always need to be done at the start of the project and some projects don’t require every step.

This is an attempt at creating a repeatable process that can work for the majority of projects.
Read More »