Find Number of Optimal Solutions in Knapsack
Comput Oper Res. 2013 Nov; 40(11): 2625–2631.
Exact solution of the robust knapsack problem ☆
Michele Monaci
aDEI, University of Padova, Via Gradenigo 6/A, I-35131 Padova, Italy
Ulrich Pferschy
bDepartment of Statistics and Operations Research, University of Graz, Universitaetsstrasse 15, A-8010 Graz, Austria
Paolo Serafini
cDIMI, University of Udine, Via delle Scienze 206, I-33100 Udine, Italy
Abstract
We consider an uncertain variant of the knapsack problem in which the weight of the items is not exactly known in advance, but belongs to a given interval, and an upper bound is imposed on the number of items whose weight differs from the expected one. For this problem, we provide a dynamic programming algorithm and present techniques aimed at reducing its space and time complexities. Finally, we computationally compare the performances of the proposed algorithm with those of different exact algorithms presented so far in the literature for robust optimization problems.
Keywords: Knapsack problem, Robust optimization, Dynamic programming
1. Introduction
The classical Knapsack Problem (KP) can be described as follows. We are given a set N = {1, …,n} of items, each of them with positive profit p j and positive weight w j , and a knapsack capacity c. The problem asks for a subset of items whose total weight does not exceed the knapsack capacity, and whose profit is a maximum. It can be formulated as the following Integer Linear Program (ILP):
Each variable x j takes value 1 if and only if item j is inserted in the knapsack.
This problem is NP-hard, although in practice fairly large instances can be solved to optimality within reasonable running time. Furthermore, dynamic programming algorithms with pseudopolynomial running time are available. A comprehensive survey on all aspects of (KP) was given by Kellerer et al. [11].
In this paper we consider the following variant of (KP), aimed at modeling uncertainties in the data, in particular in the weights: for each item j the weight may deviate from its given nominal value w j and attain an arbitrary value in some known interval . A feasible solution must obey the capacity constraint (2) no matter what the actual weight of each item turns out to be. However, uncertainty is bounded by an integer parameter Γ indicating that at most Γ items in the solution can change from their nominal value w j to an arbitrary value in the interval. Clearly a diminution of a weight below the nominal value does not affect feasibility and in the worst case all changed weights reach their upper limit. Hence a feasible solution consists of a subset of items J ⊆N such that
(4)
We call this problem the Robust Knapsack Problem (RKP). It was recently considered by Monaci and Pferschy [16] who studied the worst-case ratio between the optimal solution value of (KP) and that of (RKP), as well as the ratio between the associated fractional relaxations. A similar setting with the restriction that for all j for some given constant δ > 0 was introduced by Bertsimas and Sim [4]. Clearly this is a particular case of the model considered in this paper.
In the following we assume, without loss of generality, that all input data are integer and items are sorted according to non-increasing values. For notational simplicity we define the increased weights by for all j. In addition, for any given set S of items, we will denote by p(S) = ∑ j∈S p j and w(S) = ∑ j∈S w j the total profit and weight, respectively, of the items in S.
In this paper we review exact solution algorithms for (RKP). Although it is an NP-hard problem, exact solutions can be found in reasonable time even for large instances (see in Section 6 the computing times for instances up to 5000 items). Hence it is adequate to look for exact methods in solving (RKP) and it is interesting to compare the behavior of different algorithms. The algorithms proposed in the literature up-to-date present quite distinct features, although two of them can be shown to be very tightly intertwined.
In Section 2 we present a dynamic programming algorithm. The algorithm mimics the well known algorithm for the standard knapsack problem, but is able to take care of the upper weights once the items are sorted according to non-increasing values. This algorithm, developed by the authors, is investigated in detail. In Section 2 we show its correctness, and, as in the usual knapsack algorithm, we show that a similar version obtained by exchanging the roles of weights and values can be also formulated thus paving the way to approximation schemes.
In Section 3 we address the delicate issue of implementing the algorithm with a reduced amount of memory, since, with a large number of items and large data coefficients, space requirements can constitute a problem.
Other exact approaches are presented in Section 4. In particular we review the integer programming model by Bertsimas and Sim [4] (Section 4.1), an improvement on the iterative approach by Bertsimas and Sim [3] which requires solving O(n) knapsack instances (Section 4.2) and the Branch-and-Cut algorithm by Fischetti and Monaci [8] in which the robustness requirements are modeled by cutting inequalities (Section 4.3).
The problem we investigate is a special combinatorial optimization problem that has been motivated by a particular modeling of the problem uncertainties. By and large this is the model which has received most attention in the literature, although a lot of research has been done to face problems with uncertain data (see, e.g. the recent survey by Bertsimas, Brown and Caramanis [2]). Recently, Poss [19] pointed out drawbacks of this approach from a probabilistic point of view.
As to uncertainty in knapsack problems, few contributions were proposed. (RKP) was first introduced by Bertsimas and Sim [3]. Klopfenstein and Nace [12] defined a robust chance-constrained variant of the knapsack problem and studied the relation between feasible solutions of this problem and those of (RKP). A polyhedral study of (RKP) was conducted by the same authors in [13], where some computational experiments with small instances (up to 20 items) were given. Recently, Büsing et al. [5,6] addressed the robust knapsack problem within the so-called recoverable robustness context in which one is required to produce a solution that is not necessarily feasible under uncertainty, but whose feasibility can be recovered by means of some legal moves. In [5,6], legal moves correspond to the removal of at most K items from the solution, so as to model a telecommunication network problem. For this problem, the authors gave different ILP formulations, cut separation procedures and computational experiments.
2. A dynamic programming algorithm
In this section we present an exact dynamic programming algorithm for (RKP). Note that the same problem was considered by Klopfenstein and Nace [12] who sketched a related dynamic programming recursion in their Theorem 3. While the brief description of the algorithm in [12] relies on a modification of a dynamic program for the nominal knapsack problem, we present a detailed algorithm explicitly designed for (RKP) which allows for an improvement of the complexities. In Section 3 we will analyze time and space complexities of our algorithm, and propose possible methods for reducing both of them; finally, the algorithm will be used in Section 5 to derive a fully polynomial approximation scheme for (RKP).
Our approach is based on the following two dynamic programming arrays: Let be the highest profit for a feasible solution with total weight d in which only items in {1, …,j} ⊆N are considered and exactly s of them are included, all with their upper weight bound . Let z(d,j) be the highest profit for a feasible solution with total weight d in which only items in {1, …,j} ⊆N are considered and exactly Γ of them change from their nominal weight to their upper bound. Clearly, d = 0, 1, …,c, s = 0, 1, …,Γ, and j = 0, 1, …,n.
A crucial property for the correctness of our approach is the assumption that items are sorted by non-increasing weight increases . This implies the following lemma. For a subset of items J ⊆N denote by j Γ the index of the Γ-th item in J if |J| ≥Γ, otherwise j Γ is the index of the last item in J.
Lemma 1
A subset J ⊆N is feasible if and only if
Proof
The largest increase of w(J) caused by items attaining their upper weight is due to the subset of Γ items for which the increase is largest. If J is feasible with respect to this subset, it is feasible for any other subset of J. □
Now we can compute all array entries by the following dynamic programming recursions:
(5)
The initialization values are for d=0,…,c and s = 0, …,Γ. Then we set . The two arrays are linked together by initializing for all d. Obviously, all entries with (respectively d <w j ) are not used in definition of (respectively z) in recursion (5). The optimal solution value of the robust knapsack problem can be found as
and consumes a total capacity c ⁎ ≤c.
Intuitively, the dynamic programming algorithm operates in two phases: first, it determines the best solution consisting of (at most) Γ items with increased weight. Then, this solution is possibly extended with additional items at their nominal weight. This separation into two phases is possible because the sorting by non-increasing guarantees that in any solution the items with smallest indices, i.e. those that were packed into the knapsack earlier, are those that will attain their increased weight (see Lemma 1).
Theorem 2
The dynamic programming recursion (5) yields an optimal solution of (RKP).
Proof
We build an acyclic directed graph and show that the recursion corresponds to a longest path in the graph. The nodes are labeled as (d,s,j), with d = 0, …,c, j = 0, …,n, s = 0, …,Γ. The node (0, 0, 0) is the source and an additional node, labeled t, is the destination.
The arcs are defined as follows: within each group of nodes with the same label s (let us denote them as a "stage") there are arcs (d,s,j − 1) → (d,s,j) with value 0. Let us denote these arcs as "empty". Using an empty arc corresponds to never inserting item j. Moreover, there are other empty arcs with value 0 from each node (d,s,n) to the destination t to model situations in which the solution includes less than Γ items.
From stage s − 1 to stage s there are arcs , if , with value p j . Let us denote these arcs as "heavy". Using a heavy arc corresponds to inserting item j and assuming it will take the upper weight .
Finally, within stage s =Γ there are arcs (d −w j ,Γ,j − 1) → (d,Γ,j), if d ≥w j , with value p j . Let us denote these arcs as "light". Using a light arc corresponds to inserting item j and assuming it will take its nominal weight.
Fig. 1 shows an example of the graph associated with an instance with n=4, w = (1, 1, 3, 1), , c=8 and Γ = 2. The source and the destination are the larger white nodes. The nodes in gray are unreachable and their arcs are not shown.
Directed graph model for the dynamic programming recursion. Arcs are directed from left to right.
We claim that there is a one-to-one correspondence between paths from source to destination and feasible solutions.
Consider any path from node (0, 0, 0) to node t. Note that this path moves from a stage to the next one only by heavy arcs. In particular, it will reach the final stage only after using exactly Γ heavy arcs. Note also that each path visits the nodes in increasing order of indices j. By construction, the indices of the light arcs (if any) are greater than the indices of the heavy arcs. Hence, by Lemma 1, the solution given by the path is feasible. Of course, this is true also in case a light arc is used to go from a node (d,s,n) to the destination t; indeed, in this case, only s ≤Γ items are inserted in the solution, all of them with increased weight .
As to the reverse, take any feasible solution J, and let be its first Γ items if |J| >Γ, otherwise . As before j Γ is the last index in . We build the path as follows: for each index j ≤j Γ we choose an empty arc if j ∉J and a heavy arc if j ∈J, until we reach the node . If |J| ≤Γ we use an empty arc from this node till the destination node t. If |J| >Γ, then the node is actually , i.e. from this node we use arcs within stage Γ for all j >j Γ , in particular empty arcs if j ∉J and light arcs if j ∈J, until we reach the node and from this node we go to the destination.
Now the recursions (5) are exactly the optimality conditions for a longest path in the graph. □
3. Time and space complexity
In this section we analyze the time and space complexity of the dynamic programming algorithm resulting from (5). In many practical applications the space requirements are the main obstacle for using a dynamic programming algorithm. Hence, we will pay special attention to this issue.
A straightforward evaluation of the recursion (5) starts with array . In principle, we go through all items j from 1 to n in an outer loop. For each j, all cardinalities s ≤Γ of a solution are considered, and in the inner-most loop all capacity values d from 0 to c are evaluated. Then array z is initialized with the outcome of for s =Γ. Again, an outer loop goes over all items and an inner loop over all capacities. This would yield a pseudopolynomial running time of 𝒪(Γ nc) for computing the optimal solution value.
For the above recursion, at each iteration j only values from the previous iteration j − 1 are required. Thus we can construct a more refined implementation for computing z ⁎ as given by procedure Solve_RKP in Fig. 2. The above procedure applies a better utilization of the dynamic programming array and avoids copying array entries from one iteration to the next, yielding a space requirement 𝒪(n +Γ c) and a time complexity 𝒪(Γ nc). However, in this case it would not be possible to reconstruct the optimal solution set, but only the optimal solution value.
Dynamic programming algorithm for (RKP).
To obtain also the optimal solution set there are two straightforward possibilities. On one hand, one could store the dynamic programming arrays for all values of j, which allows a reconstruction of the solution set by backtracking; however, this procedure increases the space requirement by a factor n. On the other hand, one could store the current solution set for each entry of the dynamic programming array. An efficient approach would use a bit representation of the at most n items of each solution which yields a total space requirement of 𝒪(n +Γ c log n). But in this case it should be noted that the computation of a new entry of the dynamic programming array requires copying a solution set from a previous entry. Such a transfer of n bits cannot be done in constant time but induces an increased running time of 𝒪(Γ nc log n).
We will now present a more involved approach based on a recursive partitioning scheme. It is related to the general framework given in Pferschy [17] (see also Kellerer et al. [11, Sec. 3.3]) but requires special considerations since the conditions for directly applying the framework of [17] are not met by (RKP).
The main idea is a bipartitioning of the item set N =N 1 ∪N 2 with N 1 = {1, …,n/2} and N 2 = {n/2 + 1, …,n} (assuming n to be even). After computing the optimal solution value for the whole set N we reconstruct the optimal solution set recursively for each set N i .
We use the above recursion in a dynamic programming procedure Solve_RKP (c,Γ,N) which returns the optimal solution value of the robust knapsack problem for every capacity value d=1,…,c. In addition we also store for each array entry a counter k(d,Γ) which indicates how many items in the corresponding solution set were taken from the first half of items, i.e. from N 1. It is trivial to update this counter whenever an item from N 1 is inserted during the dynamic programming recursion. Note that items are never removed from a solution set. The counter value associated to the optimal solution value z ⁎ will be denoted by k ⁎.
In the recursion we will also use two non-robust variants of the knapsack problem. On one hand we will use the standard knapsack problem (KP) introduced above. It is well known that the optimal solution set of (KP) can be computed in 𝒪(nc) time and 𝒪(n +c) space by dynamic programming (see e.g. [11]). On the other hand we will use a cardinality constrained knapsack problem (E-kKP), where a strict cardinality bound k is added to (KP) such that
It was shown by Caprara et al. [7] that the optimal solution set of (E-kKP) can be computed in 𝒪(knc) time and 𝒪(n +kc) space by dynamic programming. Note that both of these algorithms also make use of the recursive partitioning scheme from Pferschy [17]. Moreover, they both compute the respective optimal solution values for all capacity values d=1,…,c.
Our partitioning argument is based on the following observation.
Lemma 3
If k ⁎ ≥Γ, then the optimal solution consists of the solutions of a robust knapsack problem with parameter Γ for N 1 and of an instance of (KP) for N 2 with nominal weights.
Otherwise, if k ⁎ <Γ, then the optimal solution consists of the solutions of an instance of (E-kKP) with parameter k ⁎ and increased weights for N 1 and a robust knapsack problem with parameter Γ −k ⁎ for N 2.
Proof
It was pointed out in Lemma 1 that in any feasible solution, and thus also in an optimal solution, the Γ items with lowest index, i.e. indices up to j Γ , are those which contribute an increased weight to the capacity; all remaining items with higher indices contribute their nominal weight.
If k ⁎ ≥Γ, then j Γ ∈N 1 and all items in the optimal solution belonging to N 2 (if any) contribute only their nominal weight. If k ⁎ <Γ, then j Γ ∈N 2 and all k ⁎ items in the optimal solution belonging to N 1 contribute their increased weight. The remaining at most Γ −k ⁎ weight increases are contributed by items in N 2. Note that this second case includes the possibility that the optimal solution contains less than Γ items. □
Based on Lemma 3 an algorithm for the robust knapsack problem is presented in Fig. 3.
Theorem 4
The recursive partitioning algorithm of Fig. 3 to compute an optimal solution of (RKP) can be performed in 𝒪(Γ nc) time and 𝒪(n +Γ c) space.
Proof
Using the dynamic programming scheme described above each call to Solve_RKP (c,Γ,N) requires 𝒪(Γ nc) time and 𝒪(n +Γ c) space. Moreover, summarizing the above discussion we can also perform each call to recursion (z ⁎,k ⁎,c ⁎,Γ,N) in 𝒪(Γ|N|c ⁎) time: Indeed, the main effort in the recursion for k ⁎ ≥Γ is the execution of Solve_RKP (c ⁎,Γ,N 1) requiring 𝒪(Γ|N|/2c ⁎) time and the solution of an instance of (KP) with item set N 2 which requires only 𝒪(|N|/2c ⁎) time. For k ⁎ <Γ we have to solve an instance of (E-kKP) in 𝒪(Γ|N|/2c ⁎) time and execute Solve_RKP (c ⁎,Γ −k ⁎,N 2) also in 𝒪(Γ|N|/2c ⁎) time. To report the solutions sets for (KP) resp. (E-kKP) we have to rerun the respective algorithms with capacity c 1 resp. c 2 since they compute the optimal solution sets only for the given capacity value c ⁎ but not for all capacity values. For both cases finding the combination c 1 +c 2 =c ⁎ can be done by a simple pass through the dynamic programming arrays.
Summing up the running time over all recursion levels we get a total running time of
which proves the claimed time complexity.
The space requirements are trivially bounded by 𝒪(|N| +Γ c). Note that before each call to procedure recursion all previously used space can be set free and thus reused in the next recursion level. □
Recursive partitioning algorithm to find an optimal solution set of (RKP).
Note that the results by Bertsimas and Sim [3] ensure that (RKP) can also be solved to optimality by solving n + 1 nominal knapsack problems (KP), see also Section 4.2. Since (KP) can be solved in 𝒪(nc) time and 𝒪(n +c) space this approach would require only 𝒪(n +c) space, but 𝒪(n 2 c) time. An improvement over [3] has recently been proved by Lee et al. [14]; according to this result, the number of executions of nominal (KP) can be reduced to n −Γ + 1, although this does not change the worst-case complexity of the approach.
4. Other exact approaches to (RKP)
In this section we review the other existing algorithmic approaches for the optimal solution of (RKP).
4.1. A compact MILP-formulation
The robust counterpart of an uncertain ILP was formulated by Bertsimas and Sim [4] as a Mixed ILP (MILP) by adding a polynomial number of variables and constraints. For the special case of (KP), the associated robust counterpart looks as follows:
(7)
The resulting model, referred to as BS MILP in the following, involves n binary and n + 1 continuous variables, and n + 1 constraints. Hence, from a theoretical point of view, enforcing robustness does not increase the computational complexity of the problem.
4.2. Iterative solution of the nominal problem
A different approach for a special class of ILPs was presented again by Bertsimas and Sim [3] who considered optimization problems in which all variables are binary and only the coefficients in the objective function are subject to uncertainty. In this case, it is enough to observe that an optimal solution exists in which variable ρ takes a value from a set of at most n + 1 candidates; once ρ is fixed, the resulting problem is a nominal problem in which some coefficients are changed. Hence, one can guess the "correct" value of ρ, solve n + 1 nominal problems and take the best of the associated solutions as the optimal robust solution. It is easy to see that, if one swaps the role of the objective function with that of the uncertain constraint, (RKP) immediately fits within these settings, which allows to adopt the scheme above. By solving n + 1 nominal (KP) instances, obtained by suitably changing the items' weights, one yields the optimal solution of a given (RKP) instance. The algorithm above requires 𝒪(nT) time, where 𝒪(T) denotes the computing time for solving the nominal (KP).
As already mentioned, this approach has been recently improved by Lee et al. [14], who proved that the optimal solution of (RKP) can be obtained by solving n −Γ + 1 nominal (KP) instances. In addition, this latter result was generalized by Álvarez-Miranda et al. [1] for binary problems in which uncertainty affects Γ coefficients of a linear objective function to be optimized on a polyhedral region.
4.3. Branch-and-cut algorithm
Fischetti and Monaci [8] noted that robustness, defined according to the definition of Bertsimas and Sim [3], can be enforced with no need of introducing additional variables, by working on the space of the original x j variables. To do this, one has to separate some robustness cuts, that, for the knapsack problem, have the following structure
(8)
For a general MILP, given the current solution x ⁎, the separation of robustness cuts associated with a given row requires 𝒪(n) if the x ⁎ is integer; otherwise separation can be carried out in 𝒪(n log n) time.
Note that the formulation of Section 4.1 and this Branch-and-Cut model have the same lower bounds given by the LP relaxations as can be seen from [4].
5. FPTAS and robust Knapsack
Approximation algorithms are an obvious alternative to the computation of exact solutions. In particular, the construction of a fully polynomial approximation scheme (FPTAS) for (RKP) is an interesting issue.
Note that the recursion (5) could be alternatively stated by exchanging the roles of profit and weights. Without lengthy explanations we briefly state the resulting recursion. Let y(p,s,j) be the smallest weight for a feasible solution with total profit p in which only items in {1, …,j} ⊆N are considered and exactly s of them are included, all with their upper weight bound . Let y(p,j) be the smallest weight for a feasible solution with total profit p in which only items in {1, …,j} ⊆N are considered and Γ of them change from their nominal weight to their upper bound.
The entries of the dynamic programming array can be computed in analogy to (5).
(9)
with the obvious initializations. Then the optimal solution value is given by max{p|y(p,n) ≤c}. The straightforward running time of this approach would be 𝒪(Γ n∑ j∈N p j ).
Since the running time is now pseudopolynomial in the sum of profits, one can apply a standard scaling argument (cf. [11, Sec. 2.6]) to obtain an FPTAS from our dynamic programming scheme. Indeed, using scaled profits , where p max is the largest profit of an item, one obtains an FPTAS for (RKP) with running time 𝒪(Γ n 3/ε) (neglecting the details of retrieving the corresponding solution set, cf. Section 3).
However, one could also use the iterative method from Section 4.2 to reach an FPTAS. Indeed, the iterative method can be applied also with any approximation algorithm for (KP). Again, n + 1 iterations yield a feasible solution for (RKP) and preserve the approximation ratio of the algorithm for the nominal problem. Therefore, one can take advantage of the highly tuned FPTAS for (KP) (see Kellerer and Pferschy [9,10]) with a running time complexity of 𝒪(n log n + 1/ε 3 log2(1/ε)) and obtain an FPTAS for (RKP) with 𝒪(n 2 log n +n · 1/ε 3 log2(1/ε)) running time. This will dominate the above approach for most reasonable parameter settings.
Of course, it would be possible to apply some of the technical features of [9,10] to speed up the FPTAS arising directly from the dynamic program for (RKP). However, it seems that this is a futile exercise with little hope for an overall improved running time complexity.
6. Computational experiments
In this section we computationally evaluate, on a large set of instances, the dynamic programming algorithm of Section 2 and the other exact algorithms for (RKP) described in Section 4. To the best of our knowledge, no (RKP) instances were proposed so far in the literature, but those used by Klopfenstein and Nace [13]; as already mentioned, these are however quite small instances (up to 20 items) that cannot be used to compare dynamic programming with the algorithms of Section 4. Thus, we randomly generated the problems in our testbed in the following way.
Test instances: In order to produce hard (RKP) instances, we first generated hard (KP) instances and then defined the robust weight of each item accordingly. According to Pisinger [18], hard (KP) instances were obtained by taking profits and weights as independent uniformly distributed values in a given interval; in particular, the following five classes of (KP) instances were generated:
- • UN, uncorrelated instances: p j and w j are integer values randomly chosen in [1,c];
- • WC, weakly correlated instances: w j values are integer values randomly generated in [1,c], and each p j is chosen among the integers in [max{1,w j −c/10},w j +c/10];
- • SC, strongly correlated instances: each weight w j is an integer value randomly chosen in [1,c] and the associated profit is p j =w j +c/10;
- • IC, inverse strongly correlated instances: each profit p j is an integer value randomly chosen in [1,c] and the associated weight is w j = min{c,p j +c/10};
- • SS, subset sum instances: weights w j are integer values randomly generated in [1,c] and p j =w j .
To test the performances of the algorithms with problems of different sizes, we generated instances with c=100 and n ∈ {100, 500, 1000, 5000}, and instances with n=5000 and c ∈ {1000, 5000}. For each combination of the parameters, 10 instances were generated; thus our testbed includes 300 (KP) instances.
As to uncertainty, in all the instances the robust weight of each item j was randomly generated in [w j ,c]. Finally, we considered different values of Γ, namely Γ ∈ {1, 10, 50}.
All instances (and the corresponding solutions) are available at http://www.or.deis.unibo.it/research.html.
Algorithms: We compared the following exact approaches for (RKP):
- • BS MILP , i.e. the compact formulation (7) for (RKP) as proposed by Bertsimas and Sim [4].
- • LLPP, i.e. the approach proposed by Lee et al. [14] as an improvement on Bertsimas and Sim [3], consisting in the iterative solution of nominal (KP) instances. To solve nominal (KP) instances, we run as a black box the publicly available routine combo by Martello et al. [15], which is actually considered the state-of-the-art for the exact solution of (KP) problems.
- • BC, i.e. the branch-and-cut algorithm to robust optimization recently proposed by Fischetti and Monaci [8]; in our implementation, separation is carried out at each node of the enumerative tree.
- • DP, i.e. the dynamic programming algorithm Solve_RKP presented in Section 3, but without the recursive scheme for space reduction.
All algorithms were coded in C language and run on an Intel i5-750 @ 2.67 GHz in single-thread mode. For solving both MILP (7) and all LPs during enumerative algorithm BC, the commercial solver IBM-ILOG Cplex 12.5 was used. For each instance, a time limit of 300 CPU seconds was given to each algorithm. For some classes of instances the CPU times are very small; thus, in order to obtain a reliable estimate, it was necessary to compute, for each instance, the CPU time spent while solving it 100 times and correspondingly divide the total time.
Results: Table 1 reports the outcome of our experiments on the 300 instances of our testbed for the three different values of Γ. In particular, each line in the table refers to a set of 10 instances, characterized by number of items n, capacity c and type t ∈ {UN,WC,SC,IC,SS}, and reports, for each value of Γ, the average computing time (arithmetic mean, in seconds) required by each algorithm (for unsolved instances, we counted a computing time equal to the time limit).
Table 1
Computational experiments on random (RKP) instances.
| Instances | Γ = 1 | Γ = 10 | Γ = 50 | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| n | c | Type | BS MILP | LLPP | BC | DP | BS MILP | LLPP | BC | DP | BS MILP | LLPP | BC | DP |
| 100 | 100 | UN | 0.020 | 0.000 | 0.018 | 0.000 | 0.015 | 0.000 | 0.031 | 0.000 | 0.012 | 0.000 | 0.007 | 0.000 |
| 100 | 100 | WC | 0.057 | 0.000 | 0.027 | 0.000 | 0.013 | 0.000 | 0.019 | 0.000 | 0.011 | 0.000 | 0.012 | 0.000 |
| 100 | 100 | SC | 0.096 | 0.000 | 0.052 | 0.000 | 0.029 | 0.000 | 0.077 | 0.000 | 0.017 | 0.000 | 0.032 | 0.000 |
| 100 | 100 | IC | 0.006 | 0.000 | 0.000 | 0.000 | 0.006 | 0.000 | 0.002 | 0.000 | 0.006 | 0.000 | 0.003 | 0.000 |
| 100 | 100 | SS | 0.008 | 0.000 | 0.042 | 0.000 | 0.008 | 0.000 | 0.017 | 0.000 | 0.008 | 0.000 | 0.008 | 0.000 |
| 500 | 100 | UN | 0.150 | 0.001 | 0.125 | 0.000 | 0.080 | 0.000 | 1.561 | 0.000 | 0.040 | 0.000 | 0.690 | 0.001 |
| 500 | 100 | WC | 0.493 | 0.001 | 0.357 | 0.000 | 0.084 | 0.000 | 0.302 | 0.000 | 0.047 | 0.000 | 0.237 | 0.001 |
| 500 | 100 | SC | 0.390 | 0.001 | 0.353 | 0.000 | 0.150 | 0.000 | 3.612 | 0.000 | 0.051 | 0.000 | 2.749 | 0.001 |
| 500 | 100 | IC | 0.023 | 0.001 | 0.002 | 0.000 | 0.024 | 0.000 | 0.005 | 0.000 | 0.024 | 0.000 | 0.006 | 0.001 |
| 500 | 100 | SS | 0.046 | 0.001 | 2.282 | 0.000 | 0.039 | 0.000 | 0.968 | 0.000 | 0.039 | 0.000 | 0.354 | 0.001 |
| 1000 | 100 | UN | 0.488 | 0.002 | 0.448 | 0.000 | 0.187 | 0.000 | 9.712 | 0.000 | 0.084 | 0.000 | 4.675 | 0.002 |
| 1000 | 100 | WC | 1.213 | 0.002 | 0.887 | 0.000 | 0.238 | 0.000 | 1.833 | 0.000 | 0.104 | 0.000 | 1.281 | 0.002 |
| 1000 | 100 | SC | 1.634 | 0.002 | 1.595 | 0.000 | 0.391 | 0.000 | 36.932 | 0.001 | 0.103 | 0.000 | 29.323 | 0.002 |
| 1000 | 100 | IC | 0.054 | 0.002 | 0.009 | 0.000 | 0.055 | 0.000 | 0.002 | 0.000 | 0.055 | 0.000 | 0.006 | 0.002 |
| 1000 | 100 | SS | 0.097 | 0.001 | 11.905 | 0.000 | 0.098 | 0.000 | 1.452 | 0.001 | 0.097 | 0.000 | 1.805 | 0.002 |
| 5000 | 100 | UN | 6.323 | 0.010 | 7.649 | 0.001 | 2.046 | 0.002 | 300.000 | 0.002 | 0.320 | 0.001 | 300.000 | 0.010 |
| 5000 | 100 | WC | 6.560 | 0.009 | 7.601 | 0.001 | 3.375 | 0.002 | 259.393 | 0.002 | 0.520 | 0.001 | 249.379 | 0.010 |
| 5000 | 100 | SC | 14.035 | 0.010 | 36.305 | 0.001 | 4.006 | 0.002 | 300.000 | 0.003 | 0.436 | 0.001 | 300.000 | 0.010 |
| 5000 | 100 | IC | 0.134 | 0.012 | 0.039 | 0.001 | 0.135 | 0.002 | 0.037 | 0.002 | 0.134 | 0.001 | 0.038 | 0.008 |
| 5000 | 100 | SS | 0.468 | 0.007 | 101.486 | 0.001 | 0.464 | 0.002 | 30.799 | 0.003 | 0.316 | 0.001 | 36.006 | 0.010 |
| 5000 | 1000 | UN | 8.094 | 0.092 | 8.760 | 0.006 | 4.064 | 0.019 | 300.000 | 0.016 | 0.455 | 0.010 | 300.000 | 0.052 |
| 5000 | 1000 | WC | 7.369 | 0.092 | 9.981 | 0.006 | 3.926 | 0.017 | 300.000 | 0.017 | 0.432 | 0.010 | 300.000 | 0.052 |
| 5000 | 1000 | SC | 79.488 | 0.106 | 66.597 | 0.006 | 24.095 | 0.017 | 300.000 | 0.018 | 1.346 | 0.010 | 300.000 | 0.054 |
| 5000 | 1000 | IC | 0.143 | 0.116 | 0.040 | 0.006 | 0.144 | 0.028 | 0.039 | 0.013 | 0.143 | 0.011 | 0.039 | 0.042 |
| 5000 | 1000 | SS | 0.522 | 0.061 | 300.000 | 0.008 | 0.459 | 0.018 | 300.000 | 0.019 | 0.351 | 0.011 | 300.000 | 0.054 |
| 5000 | 5000 | UN | 8.293 | 0.289 | 8.695 | 0.031 | 3.765 | 0.075 | 300.000 | 0.077 | 0.449 | 0.035 | 300.000 | 0.240 |
| 5000 | 5000 | WC | 10.475 | 0.289 | 10.751 | 0.031 | 4.562 | 0.070 | 300.000 | 0.079 | 0.481 | 0.034 | 300.000 | 0.242 |
| 5000 | 5000 | SC | 131.528 | 0.403 | 107.186 | 0.031 | 16.599 | 0.069 | 300.000 | 0.085 | 1.173 | 0.033 | 300.000 | 0.248 |
| 5000 | 5000 | IC | 0.143 | 0.630 | 0.037 | 0.028 | 0.145 | 0.139 | 0.038 | 0.060 | 0.143 | 0.041 | 0.036 | 0.193 |
| 5000 | 5000 | SS | 1.397 | 0.297 | 300.000 | 0.037 | 0.867 | 0.158 | 300.000 | 0.085 | 0.727 | 0.053 | 291.023 | 0.248 |
| Avg | 9.325 | 0.081 | 32.774 | 0.007 | 2.336 | 0.021 | 111.561 | 0.016 | 0.271 | 0.008 | 110.590 | 0.050 | ||
Computational results show that our dynamic programming algorithm is very effective in solving (RKP) instances where a small number of coefficients change, as its complexity grows linearly with Γ. For larger values of Γ the iterative solution of (KP) instances, as proposed in [3,14] turns out to be the best algorithmic approach. However, the computing time of this latter approach may vary in a substantial way when solving instances of different type – although of the same size – which is not the case for algorithm DP, whose computing time is more stable. This is particularly evident when instances with c=5000 are considered.
The two algorithms above outperform the remaining two algorithms, BS MILP and BC. This is not surprising, as the latter are based on the use of a general purpose ILP solver, which is usually less efficient than ad hoc algorithms for knapsack problems. In addition, cutting planes are not the best suitable way to enforce robustness when problems with integer variables are considered, as noted by Fischetti and Monaci [8], which leads to a number of unsolved instances in our testbed.
7. Conclusions
In this paper we considered the robust knapsack problem, i.e. the uncertain variant of the well-known knapsack problem in which robustness is enforced according to the Bertsimas-Sim definition (see [4]). For this problem we presented a dynamic programming algorithm and studied its time and space complexities, as well as techniques aimed at reducing the space complexity. We then computationally tested this algorithm on a large set of randomly generated instances, and compared the performances of the algorithm with those of other exact approaches proposed in the literature for (RKP). Computational results showed that dynamic programming is a viable way for solving (RKP) even for large-sized instances.
Acknowledgments
We would like to thank an anonymous referee for helpful remarks and suggestions to improve the presentation of the paper.
Ulrich Pferschy was supported by the Austrian Science Fund (FWF): [P 23829-N13] "Graph Problems with Knapsack Constraints".
Footnotes
☆This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-No Derivative Works License, which permits non-commercial use, distribution, and reproduction in any medium, provided the original author and source are credited.
References
1. Álvarez-Miranda E, Ljubic I, Toth P. A note on the Bertsimas & Sim algorithm for robust combinatorial optimization problems. 4OR. In press, http://dx.doi.org/10.1007/s10288-013-0231-6.
2. Bertsimas D., Brown D.B., Caramanis C. Theory and applications of robust optimization. SIAM Review. 2011;53:464–501. [Google Scholar]
3. Bertsimas D., Sim M. Robust discrete optimization and network flows. Mathematical Programming. 2003;98:49–71. [Google Scholar]
4. Bertsimas D., Sim M. The price of robustness. Operations Research. 2004;52:35–53. [Google Scholar]
5. Büsing C, Koster AMCA, Kutschka M. Recoverable robust knapsacks: Γ-scenarios. In: Proceedings of INOC 2011, international network optimization conference, lecture notes in computer science 6701. Springer; 2011. pp. 583–8.
6. Büsing C., Koster A.M.C.A., Kutschka M. Recoverable robust knapsacks: the discrete scenario case. Optimization Letters. 2011;5:379–392. [Google Scholar]
7. Caprara A., Kellerer H., Pferschy U., Pisinger D. Approximation algorithms for knapsack problems with cardinality constraints. European Journal of Operational Research. 2000;123:333–345. [Google Scholar]
8. Fischetti M., Monaci M. Cutting plane versus compact formulations for uncertain (integer) linear programs. Mathematical Programming Computation. 2012;4:239–273. [Google Scholar]
9. Kellerer H., Pferschy U. A new fully polynomial approximation scheme for the knapsack problem. Journal of Combinatorial Optimization. 1999;3:59–71. [Google Scholar]
10. Kellerer H., Pferschy U. Improved dynamic programming in connection with an FPTAS for the knapsack problem. Journal of Combinatorial Optimization. 2004;8:5–12. [Google Scholar]
11. Kellerer H., Pferschy U., Pisinger D. Springer; 2004. Knapsack problems. [Google Scholar]
12. Klopfenstein O., Nace D. A robust approach to the chance-constrained knapsack problem. Operations Research Letters. 2008;36:628–632. [Google Scholar]
13. Klopfenstein O., Nace D. Cover inequalities for robust knapsack sets—application to the robust bandwidth packing problem. Networks. 2012;59:59–72. [Google Scholar]
14. Lee C., Lee K., Park K., Park S. Technical note—branch-and-price-and-cut approach to the robust network design problem without flow bifurcations. Operations Research. 2012;60:604–610. [Google Scholar]
15. Martello S., Pisinger D., Toth P. Dynamic programming and strong bounds for the 0–1 knapsack problem. Management Science. 1999;3:414–424. [Google Scholar]
16. Monaci M, Pferschy U. On the robust knapsack problem. Submitted manuscript, available as Optimization Online 2011-04-3019; 2012.
17. Pferschy U. Dynamic programming revisited: improving knapsack algorithms. Computing. 1999;63:419–430. [Google Scholar]
18. Pisinger D. Where are the hard knapsack problems? Computers and Operations Research. 2005;32:2271–2284. [Google Scholar]
19. Poss M. Robust combinatorial optimization with variable budgeted uncertainty. 4OR. 2013;11:75–92. [Google Scholar]
Find Number of Optimal Solutions in Knapsack
Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3727332/