differentiable kdes #77
Conversation
|
the following now works: using Flux,KernelDensity
data = param(randn(100))
xrange = -3:0.1:3
kde(data,xrange,bandwidth=0.1) |
| return dropdims(result, dims = (1+N,2+N)) | ||
| end | ||
|
|
||
| # patches for TrackedReal and Vector{TrackedReal} |
There was a problem hiding this comment.
all of these methods below are type piracy, so probably better to make a PR to Tracker.jl
There was a problem hiding this comment.
I see.. suppose these methods are added to Tracker.jl how would you rewrite the conv methods in univariate.jl and bivariate.jl so that they are compatible?
| """n-dimensional convolution""" | ||
| function conv(x::AbstractArray{T,N}, w::AbstractArray{T,N}) where {T,N} | ||
| wdim = Int.(ceil.((size(w).-1)./2)) | ||
| padding = Iterators.flatten([ (wdim[i],wdim[i]) for i=1:length(wdim) ]) |> collect | ||
|
|
||
| dims = DenseConvDims((size(x)...,1,1),(size(w)...,1,1); padding=padding ) | ||
| result = Tracker.conv( reshape(x,(size(x)...,1,1)), reshape(w,(size(w)...,1,1)), dims) | ||
| return dropdims(result, dims = (1+N,2+N)) | ||
| end | ||
|
|
||
| # patches for TrackedReal and Vector{TrackedReal} | ||
| conv(x::AbstractArray{TrackedReal{T},N}, w::AbstractArray) where {T,N} = conv(Tracker.collect(x),w) | ||
| conv(x::AbstractArray, w::AbstractArray{TrackedReal{T},N}) where {T,N} = conv(x,Tracker.collect(w)) | ||
| conv(x::AbstractArray{TrackedReal{T},N}, w::AbstractArray{TrackedReal{T},N}) where {T,N} = conv(Tracker.collect(x),Tracker.collect(w)) | ||
|
|
||
| round(::Type{R}, t::TrackedReal) where {R<:Real} = round(R, t.data) | ||
| round(t::TrackedReal, mode::RoundingMode) = round(t.data, mode) | ||
| Integer(x::TrackedReal) = Integer(x.data) | ||
|
|
There was a problem hiding this comment.
bypasses the need for FFTW which is not compatible with tracked types (JuliaMath/AbstractFFTs.jl#35). Tracker.conv throws outofMemoryError() for large kernels w which is why tests are failing
| density::Matrix{Float64} | ||
| density::AbstractMatrix{} |
There was a problem hiding this comment.
relax type casting for future compatibility.. will make a PR to Tracker.jl soon
| grid = zeros(Float64, nx, ny) | ||
| grid = zeros(eltype(xdata),nx,ny) |
| function conv(k::UnivariateKDE, dist::UnivariateDistribution) | ||
| # Transform to Fourier basis | ||
| K = length(k.density) | ||
| ft = rfft(k.density) | ||
|
|
||
| # Convolve fft with characteristic function of kernel | ||
| # empirical cf | ||
| # = \sum_{n=1}^N e^{i*t*X_n} / N | ||
| # = \sum_{k=0}^K e^{i*t*(a+k*s)} N_k / N | ||
| # = e^{i*t*a} \sum_{k=0}^K e^{-2pi*i*k*(-t*s*K/2pi)/K} N_k / N | ||
| # = A * fft(N_k/N)[-t*s*K/2pi + 1] | ||
| c = -twoπ/(step(k.x)*K) | ||
| for j = 0:length(ft)-1 | ||
| ft[j+1] *= cf(dist,j*c) | ||
| end | ||
|
|
||
| dens = irfft(ft, K) | ||
| # fix rounding error. | ||
| for i = 1:K | ||
| dens[i] = max(0.0,dens[i]) | ||
| end | ||
|
|
||
| # Invert the Fourier transform to get the KDE | ||
| UnivariateKDE(k.x, dens) | ||
| half_grid = range(step(k.x),5*std(dist),step=step(k.x)) | ||
| grid = [-reverse(half_grid);0;half_grid] | ||
| density = conv(k.density, pdf.(dist,grid)) * step(k.x) | ||
| UnivariateKDE(k.x, density) | ||
| end |
There was a problem hiding this comment.
what would your recommendation be for this? given that the methods in KernelDensity.jl are type piracy?
|
closing this request since all changes have been moved into patches |
|
How are those patches meant to be applied? |
|
download that file and instead of |
|
note that currently the Fourier transform for |
I would like the
kdemethod to be used as part of a machine learning pipline. Therefore we need all methods to be extended toTrackedArraytypes. The plan would be to extendunivariate.jlfirst. Which can be done by by-passing the use ofrfft,irfftfromFFTW.jl(which is not compatible with TrackedArrays) and use theconvlayer fromFlux.jlcloses #76