Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running a conduit in a function with an unrelated parameter twice leaks memory #511

Open
matil019 opened this issue Jun 14, 2024 · 0 comments

Comments

@matil019
Copy link

Troubleshooting my own program, I stumbled upon another instance of memory leak.

When you call twice a function which ignores its parameter, and the function runs a conduit, it leaks memory.

This is my minimized example:

#!/usr/bin/env cabal
{- cabal:
build-depends:    base ^>=4.17.2.1
                , conduit ^>=1.3.5
default-language: GHC2021
ghc-options: -with-rtsopts=-M500M -threaded
-}
{- project:
with-compiler: ghc-9.4
-}

import Data.Conduit ((.|), runConduit)
import Data.Conduit.Combinators qualified as CC

main :: IO ()
main = do
  _ <- go ()
  _ <- go ()
  pure ()
  where
  go _ = runConduit
     $ CC.yieldMany [(1 :: Integer)..]
    .| CC.last

The above program exits with "Heap exhausted;" after a while thanks to the rtsopts.

Here is what I observed so far:

  • Not only CC.lastDef but also CC.last causes memory leak (unlike Functor instance causes memory leak when used with Shake #510)
  • Moving go to the top level still causes memory leak
  • Using the parameter but irrelevantly with the conduit still causes memory leak (see below)
  • Replacing [(1 :: Integer)..] with iterate' succ (1 :: Integer) still causes memory leak
  • Calling go only once does not leak memory
  • Removing the paremeter altogether does not leak memory
  • Using the parameter as part of CC.yieldMany does not leak memory (see below)

Just evaluating the parameter but irrelevantly with the conduit still causes the memory leak:

-- Leaks memory, aborts
main :: IO ()
main = do
  _ <- go ()
  _ <- go ()
  pure ()
  where
  go x = do
    print x
    runConduit
       $ CC.yieldMany [(1::Integer)..]
      .| CC.last

But using the parameter for CC.yieldMany prevents memory leak:

-- Doesn't leak memory, loops forever
main :: IO ()
main = do
  let i = 1 :: Integer
  _ <- go i
  _ <- go i
  pure ()
  where
  go i = runConduit
     $ CC.yieldMany [i..]
    .| CC.last

I have completely no idea what is going on. Please let me know if I misunderstand the evaluation strategy of Haskell.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant