Heapsort

  • Running time is O(nlgn).
  • Sorts in place, only a constant number of elements needed in addition to the input.
  • Manages data with a heap.

A binary heap can be represented as a binary tree, but is stored as an array. The root is the first element of the array. The left subnode for the element at index i is located at 2i and the right subnode is located at 2i+1. This assumes a 1-based indexing.

Using 0-based indexing, we can use 2i+1 for the left and 2i+2 for the right. The parent could be accessed via i12.

<span class="figure-number">Figure 1: </span>A binary tree as a heap with its array representation (<a href="#citeproc_bib_item_1">Cormen et al. 2022</a>).

Figure 1: A binary tree as a heap with its array representation (Cormen et al. 2022).

Heaps come in two flavors: max-heaps and min-heaps. They can be identified by satisfying a heap property.

  1. max-heap property: A[parent(i)]A[i]
  2. min-heap property: A[parent(i)]A[i]

These properties imply that the root is the largest element in a max-heap and the smallest element in a min-heap.

When it comes to heapsort, a max-heap is used. Min-heaps are used in priority queues. These notes will cover both.

Maintaining the Heap Property

When using heapsort, the heap should always satisfy the max-heap property. This relies on a procedure called max_heapify. This function assumes that the root element may violate the max-heap property, but the subtrees rooted by its subnodes are valid max-heaps. The function then swaps nodes down the tree until the misplaced element is in the correct position.

def max_heapify(A, i, heap_size):
    l = left(i)
    r = right(i)
    largest = i
    if l < heap_size and A[l] > A[i]:
        largest = l
    if r < heap_size and A[r] > A[largest]:
        largest = r
    if largest != i:
        A[i], A[largest] = A[largest], A[i]
        max_heapify(A, largest, heap_size)

Analysis of Max Heapify

Given that max_heapify is a recursive function, we can analyze it with a recurrence. The driving function in this case would be the fix up that happens between the current node and its two subnodes, which is a constant time operation. The recurrence is based on how many elements are in the subheap rooted at the current node.

In the worst case of a binary tree, the last level of the tree is half full. That means that the left subtree has height h+1 compared to the right subtree’s height of h. For a tree of size n, the left subtree has 2h+21 nodes and the right subtree has 2h+11 nodes. This is based on a geometric series.

We now have that the number of nodes in the tree is equal to 1+(2h+21)+(2h+11).

n=1+2h+21+2h+11n=2h+2+2h+11n=2h+1(2+1)1n=32h+11

This implies that 2h+1=n+13. That means that, in the worst case, the left subtree would have 2h+21=2(n+1)31 nodes which is bounded by 2n3. Thus, the recurrence for the worst case of max_heapify is T(n)=T(2n3)+O(1).

Building the Heap

Given an array of elements, how do we build the heap in the first place? The solution is to build it using a bottom-up approach from the leaves. The elements from n2+1 to n are all leaves. This means that they are all 1-element heaps. We can then run max_heapify on the remaining elements to build the heap.

def build_max_heap(A):
    heap_size = len(A)
    for i in range(len(A) // 2, -1, -1):
        max_heapify(A, i, heap_size)

Why does this work?

Each node starting at n2+1 is the root of a 1-element heap. The subnodes, which are to the right of node n2, are roots of their own max-heaps. The procedure loops down to the first node until all sub-heaps have been max-heapified.

The figure below is from (Cormen et al. 2022) and shows the process of building a max-heap from an array.

<span class="figure-number">Figure 2: </span>Building a max-heap from an array (<a href="#citeproc_bib_item_1">Cormen et al. 2022</a>).

Figure 2: Building a max-heap from an array (Cormen et al. 2022).

Analysis of Build Max Heap

A general analysis is fairly straightforward considering that the call to max_heapify is O(lgn). The loop in build_max_heap runs O(n) times. This means that the overall running time is O(nlgn). A more careful analysis can be done by considering the height of the tree and the number of nodes at each level.

A heap of n elements has height lgn Each call to max_heapify can also be viewed in terms of the height of the tree h, so the upper bound is O(h). This bounds build_max_heap at h=0lgnn2h+1ch. When h=0, the first term n2h+1=n2. When h=lgn, n2h+1=1. Thus, n2h+112 for 0hlgn.

Let x=n2h+1. Since x12, we have that x2x. This means that n2h+12n2h+1=n2h. An upper bound can now be derived.

h=0lgnn2h+1chh=0lgnn2hch=cnh=0lgnh2hcnh=0h2hcn1/2(11/2)2(See CRLS for details)=O(n)

Thus, a heap can be constructed in linear time. This is independent on whether the original data is already sorted.

Heapsort

We now have all of the components necessary to implement heapsort. The algorithm is as follows:

def heapsort(A):
    build_max_heap(A)
    heap_size = len(A)
    for i in range(len(A) - 1, 0, -1):
        A[0], A[i] = A[i], A[0]
        heap_size -= 1
        max_heapify(A, 0, heap_size)

It starts by building a max-heap on the input array. As seen in the previous section, this is done in linear time. From there, it’s a matter of taking the root element out of the heap and then running max_heapify to maintain the max-heap property. This is done n1 times, so the overall running time is O(nlgn).

<span class="figure-number">Figure 3: </span>Heapsort in action (<a href="#citeproc_bib_item_1">Cormen et al. 2022</a>).

Figure 3: Heapsort in action (Cormen et al. 2022).

Heapsort is visualized in the figure above, starting with a constructed max-heap in (a) (Cormen et al. 2022).

Questions

  1. What is the running time of heapsort given an array that is already sorted in ascending order?
  2. What is the running time of heapsort given an array that is already sorted in descending order?

References

Cormen, Thomas H., Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2022. Introduction to Algorithms. 4th ed. MIT Press. http://mitpress.mit.edu/9780262046305/introduction-to-algorithms/.