Re: dlmalloc

On 15 Dec 2003, at 23:43, Henrik Nordstrom <hno@squid-cache.org> wrote:

> On Mon, 15 Dec 2003, Andres Kroonmaa wrote:
>
> > So far dlmalloc has been very good malloc. Why is it not safe anymore?
>
> It has never been really safe and this has bitten us to various degree
> several times in the past. The switch to C++ makes this some orders of
> magnitude more complex due to additional dependencies on memory
> management. Using dlmalloc makes assumptions on many things regarding how
> the C and C++ libraries interacts with the memory management. There is no
> public defined standard on how such interactions take place an is in the
> "implementation specific" details of both C, C++ and UNIX standards.

 I'd probably trust your judgement, but I still don't understand in what
 way is it unsafe. To me it seems that optimising for typical memory usage
 patterns belongs to pretty normal assumptions.

 Could you list some requirements for malloc implementation that needs
 to be met to be considered optimal and safe malloc for Squid?

 But most interesting to me is in what ways has it bitten us? dlmalloc
 was written for C++ in the first place.. And linux glibc malloc is
 based on dlmalloc/ptmalloc afaik. Thats why hearing that it is unsafe
 is especially intriguing.

  http://gee.cs.oswego.edu/dl/html/malloc.html
  http://www.malloc.de/en/index.html

> > Instead of dropping dlmalloc, perhaps upgrade it from ver 2.6.4 (1996) to
> > ver. 2.7.2 (2002)?
>
> I don't really see why we should keep bundling a whole malloc
> implementation with Squid these days. If the OS cannot provide reasonable
> core services such as malloc then it is not a good OS in my opinion.

 This is quite farfetching really. Raises questions about why should Squid
 be portable at all. There are not many good OS'es depending on opinions..

 Also, definition of reasonable varies very largely. Most generic mallocs
 assume that application is shortlived and doesn't use huge amounts of
 small allocations. Squid blows away most malloc assumptions.

 I have understood that we bundle dlmalloc because it fits well enough
 for Squid and can be used as alternate malloc on OS'es that would suffer
 if using native libs, so that instead of forcing users into an OS hunt
 we can switch over to dlmalloc transparently. Having one source bundled
 allows us better integrate expectations of both. dlmalloc is very tunable,
 at both compiletime and runtime. This has been ignored so far.

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Data AS
 Tel: 6501 731, Fax: 6501 725
 Pärnu mnt. 158, Tallinn
 11317 Estonia
Received on Wed Dec 17 2003 - 00:39:47 MST