mini.pick file picker with sorting from mini.visits

[JulesNP]

I wanted to write a frecency file picker which shows all files which would normally be seen in :Pick files, but sorted by frecency using data from mini.visits.

I'm currently using the implementation below, but it currently doesn't work with files with uppercase letters in their name, as all the paths passed in stritems to match() seem to be lowercase. As a result, they don't match the table keys from MiniVisits.get_index().

What would be a good workaround for this?

MiniPick.registry.frecency = function()
    local visits_index = MiniVisits.get_index()[vim.loop.cwd()]
    if visits_index == nil then
        MiniPick.builtin.files()
    else
        local current_file = vim.fn.expand "%:p"
        MiniPick.builtin.files(nil, {
            source = {
                match = function(stritems, indices, query)
                    local prompt = vim.pesc(table.concat(query))
                    local res = {}

                    for _, index in ipairs(indices) do
                        local path = stritems[index]
                        local match_score = prompt == "" and 0 or MiniFuzzy.match(prompt, path).score
                        if match_score >= 0 then
                            local full_path = vim.fn.fnamemodify(path, ":p")
                            -- NOTE: Currently doesn't match files with uppercase characters
                            local visits = visits_index[full_path]
                            table.insert(res, {
                                count = visits == nil and 0 or visits.count,
                                index = index,
                                latest = visits == nil and 0 or visits.latest,
                                score = full_path == current_file and 9999 or match_score, -- Score current file last
                            })
                        end
                    end

                    local function set_rank(key)
                        local last, ties = -1, 0
                        for index, item in ipairs(res) do
                            local value = item[key]
                            ties = last == value and ties + 1 or 0
                            last = value
                            item[key] = index - ties - 1
                        end
                    end

                    table.sort(res, function(a, b)
                        return a.count > b.count
                    end)
                    set_rank "count"

                    table.sort(res, function(a, b)
                        return a.latest > b.latest
                    end)
                    set_rank "latest"

                    table.sort(res, function(a, b)
                        return (a.score + a.count + a.latest) < (b.score + b.count + b.latest)
                    end)

                    return vim.tbl_map(function(val)
                        return val.index
                    end, res)
                end,
            },
        })
    end
end

[echasnovski]

That is certainly an interesting idea to implement. Nice job trying to implement it!

Sorry, it is a bit convoluted to review in full, but here are the steps I'd take to implement this:

• Use MiniVisits.list_paths() directly to get an array of paths already sorted in the order you'd like.
• Invert it so that paths are keys and their id is a value.
• Update match() to first check that table and possibly sort based on it.

I'm currently using the implementation below, but it currently doesn't work with files with uppercase letters in their name, as all the paths passed in stritems to match() seem to be lowercase. As a result, they don't match the table keys from MiniVisits.get_index().

That depends on the query and 'ignorecase'/'smartcase' options. If you need to compare to the data from 'mini.visits', then possibly convert that to lowercase (if 'mini.pick' uses it in that particular call).

[JulesNP]

Thanks! That makes sense, and would cut out a lot of the extra scoring logic in my implementation.

[JulesNP]

@Susensio, can you advise what issue you're having with spaces in the query? For me entering a space filters the list to only files with spaces in the name. (Which is usually none for most of my codebases)

[Susensio]

Hey @JulesNP , sorry for the late reply.
My problem is, as stated previously, that special characters behave differently in MiniFuzzy. With the default file picker, spaces do not match directly, and I cannot get dots to match with MiniFuzzy.
Maybe @echasnovski can give us some indications on the differences between MiniPick matcher and MiniFuzzy matcher. How can we make them behave consistently?

[echasnovski]

Maybe @echasnovski can give us some indications on the differences between MiniPick matcher and MiniFuzzy matcher. How can we make them behave consistently?

There is no easy way, unfortunately (if at all). I initially planned for matching in 'mini.pick' to be the same as in 'mini.fuzzy'. In the end, I got a bit carried away when saw that implementing extra rules (like ^, $, ', and special treatment of spaces) has more benefits than added complexity.

I may look at having 'mini.fuzzy' behave the same as in 'mini.pick' in the future. Doubt that it will happen soon, though.

[Susensio]

I'm trying to understand the logic behind your matching algorithm. Can I ask you a couple of questions?
1 - How does MiniPick handles spaces?
2 - Why can't I match a dot . with MiniFuzzy?

[echasnovski]

1 - How does MiniPick handles spaces?

Basically, item's string should match each non-whitespace part exactly and in that particular order. So query ab c will match _ab__c but not _a_b_c.

Here us a full description of default matching with relevant example.. Here is also a short one (does not mention spaces, though).

2 - Why can't I match a dot . with MiniFuzzy?

You can. Try :=require('mini.fuzzy').filtersort('.', { 'abc', 'ab.c', 'a.bc' }).

[Susensio]

1 - How does MiniPick handles spaces?

Basically, item's string should match each non-whitespace part exactly and in that particular order. So query ab c will match _ab__c but not _a_b_c.

Here us a full description of default matching with relevant example.. Here is also a short one (does not mention spaces, though).

I see. Is there any way of calling default_match without sorting?

2 - Why can't I match a dot . with MiniFuzzy?

You can. Try :=require('mini.fuzzy').filtersort('.', { 'abc', 'ab.c', 'a.bc' }).

Okey, the problem was that vim.pesc was being executed twice, once in my code and other in MiniFuzzy.

---

Any way, I've done a rewrite of this. To improve consistency, I've managed to use MiniPick.default_match for filtering, and MiniVisits.list_paths for sorting.
That is, always sort by frecency. This drops the MiniFuzzy dependency.
@JulesNP Maybe you find this valuable. It does not have any problem with spaces nor dots (as far as I've tested).

      MiniPick.registry.frecency = function()
        local inf = math.huge

        local visit_paths = MiniVisits.list_paths()
        visit_paths = vim.tbl_map(function(path) return vim.fn.fnamemodify(path, ":.") end, visit_paths)
        vim.tbl_add_reverse_lookup(visit_paths)

        -- current file last
        local current_file = vim.fn.expand("%:.")
        if visit_paths[current_file] then
          visit_paths[current_file] = inf
        end

        MiniPick.builtin.files(nil, {
          source = {
            name = "Files (MRU)",
            match = function(stritems, indices, query)
              local filtered = MiniPick.default_match(stritems, indices, query, true) or {}

              table.sort(filtered,
                function(item1, item2)
                  local path1 = stritems[item1]
                  local path2 = stritems[item2]
                  local score1 = visit_paths[path1] or inf    -- inf for paths not visited
                  local score2 = visit_paths[path2] or inf    -- inf for paths not visited
                  return score1 < score2
                end
              )

              return filtered
            end,
          },
        })
      end

I don't know how could this behave with a huge amount of files, as I'm calling default_match synchronously. Maybe @echasnovski can give us some directions on a better integration.

[echasnovski]

I see. Is there any way of calling default_match without sorting?

Not at the moment, no. The best alternative is to do a synchronous call and resort later.

I don't know how could this behave with a huge amount of files, as I'm calling default_match synchronously. Maybe @echasnovski can give us some directions on a better integration.

In my experience, on modern hardware anything under 10000 items should be fine. So I'd stick with doing synchronous match with later sorting.

[Susensio]

Thanks for your help!
Would you be interested in making this a Mini.extra picker? I could try to do a proper PR.

[echasnovski]

Thanks for the offer, but I think having MiniExtra.pickers.visits is enough.

[Susensio]

Got it. The problem I had with MiniExtra.pickers.visits is that it does not show unvisited files, so it cannot be the one and only file picker.

[wiseodd]

Thanks for sharing @Susensio! I can report that it works great. I would replace the default sorting algorithm with the z-like sorting that is also available in the plugin:

...
local sort = MiniVisits.gen_sort.z()
local visit_paths = MiniVisits.list_paths(nil, { sort = sort })
...

I find it behaves more similarly to telescope smart-open, which is, dare I say, the gold standard for this "omnibar"-style picker.