part2

Transcript

part2

Outline
A Structured Benchmark Domain
Randomization
Conclusions
Gomes, Hogg, Walsh, Zhang
117
IJCAI-01 Tutorial on Phase
Transitions and Structures-SA4
Outline
Randomization
Conclusions
118
Quasigroups or Latin Squares:
An Abstraction for Real World Applications
Given an N X N matrix, and given N colors, a
quasigroup of order N is a a colored matrix,
such that:
-all cells are colored.
- each color occurs exactly once in each
row.
- each color occurs exactly once in each
column.
Quasigroup or Latin Square
(Order 4)
119
Quasigroup Completion
Problem (QCP)
Given a partial assignment of colors (10 colors in
this case), can the partial quasigroup (latin square)
be completed so we obtain a full quasigroup?
Example:
32% preassignment
Gomes, Hogg, Walsh,
Zhang&
(Gomes
Selman 97)
120
Quasigroup Completion Problem
A Framework for Studying Search
NP-Complete.
Has a structure not found in random instances,
such as random K-SAT.
Leads to interesting search problems when
structure is perturbed (more about it later).
Good abstraction for several real world
problems: scheduling and timetabling, routing
in fiber optics, coding, etc
(Anderson 85, Colbourn 83, 84, Denes & Keedwell 94, Fujita et al. 93,
Gent et al. 99, Gomes & Selman 97, Gomes et al. 98, Meseguer & Walsh
98, Stergiou and Walsh 99, Shaw et al. 98, Stickel 99, Walsh 99 )
121
Fiber Optic Networks
Nodes
connect point to point
fiber optic links
122
Nodes
connect point to point
fiber optic links
Each fiber optic link supports a
large number of wavelengths
Nodes are capable of photonic switching
--dynamic wavelength routing -which involves the setting of the wavelengths.
123
Routing in Fiber Optic Networks
preassigned channels
Input Ports
1
Output Ports
1
2
2
3
3
4
4
Routing Node
How can we achieve conflict-free routing in each node of the network?
IJCAI-01
Tutorial on Phase
Dynamic wavelength routing is a NP-hard
problem.
124
QCP Example Use: Routers in
Dynamic wavelength routing in Fiber Optic Networks can be
directly mapped into the Quasigroup Completion Problem.
•each channel cannot be repeated in the same input port
(row constraints);
• each channel cannot be repeated in the same output
port (column constraints);
1
2
3
4
Output Port
1
2
3
4
Input ports
Input Port
Output ports
CONFLICT FREE
LATIN ROUTER
(Barry and Humblet 93, Cheung et al. 90, Green 92, Kumar et al. 99)
125
NP-completeness is a worstcase notion – what about average
complexity?
Structural differences
between instances of the same
NP- complete problem (QCP)
Are all the Quasigroup Instances
(of same size) Equally Difficult?
Time performance:
150
1820
165
IJCAI-01
Tutorial on Phase
What
is Walsh,
the Zhang
fundamental difference between
instances?
Gomes, Hogg,
127
Are all the Quasigroup Instances
Equally Difficult?
Time performance:
150
Fraction of preassignment:
1820
165
40%
50%
35%
128
Median Runtime (log scale)
Complexity of Quasigroup
Completion
Critically constrained area
Underconstrained
area
20%
Overconstrained area
42%
50%
Fraction of pre-assignment
129
Complexity
Graph
Phase
Transition
Fraction of unsolvable cases
Phase transition
from almost all solvable
to almost all unsolvable
Almost all solvable
area
Almost all unsolvable
area
Fraction of pre-assignment
130
These results for the QCP - a structured
domain, nicely complement previous results on
phase transition and computational complexity
for random instances such as SAT, Graph
Coloring, etc.
(Broder et al. 93; Clearwater and Hogg 96, Cheeseman et al. 91, Cook and
Mitchell 98, Crawford and Auton 93, Crawford and Baker 94, Dubois 90,
Frank et al. 98, Frost and Dechter 1994, Gent and Walsh 95, Hogg, et al. 96,
Mitchell et al. 1992, Kirkpatrick and Selman 94, Monasson et 99, Motwani
et al. 1994, Pemberton and Zhang 96, Prosser 96, Schrag and Crawford
96, Selman and Kirkpatrick 97, Smith and Grant 1994, Smith and Dyer 96,
Zhang and Korf 96, and more)
131
QCP
Different Representations /
Encodings
Rows
Colors
Columns
Cubic representation of QCP
133
QCP as a MIP
O(n3)
cell i, j has color k; i, j,k =1, 2, ...,n.
• Variables -
x
ijk
x ∈ {0,1}
ijk
• Constraints - O(n2)
Row/color line
∀
x ≤ 1 i, j,k =1, 2, ...,n.
∑
j,k
ijk
i
Column/color line
∀ ∑ x ≤ 1 i, j,k =1, 2, ...,n.
i,k
ijk
j
Row/column line
∀ , ∑ x ≤ 1 i, j,k =1, 2, ...,n.
i, j
ijk
k
134
QCP as a CSP
• Variables -
O(n2) [ vs. O(n3) for MIP]
x color of cell i, j; i, j =1, 2, ...,n.
i, j
x ∈ {1, 2, ...,n}
i, j
• Constraints - O(n)
[ vs. O(n2) for MIP]
alldiff (x , x ,..., x ); i =1, 2, ...,n.
i,n
i,1 i,2
row
alldiff (x , x ,..., x ); j =1, 2, ...,n. column
n, j
1, j 2, j
135
Exploiting Structure for Domain
Reduction
• A very successful strategy for domain
reduction in CSP is to exploit the structure
of groups of constraints and treat them as
global constraints.
Example using Network Flow Algorithms:
• All-different constraints
(Caseau and Laburthe 94, Focacci, Lodi, & Milano 99, Nuijten & Aarts 95,
Ottososon & Thorsteinsson 00, Refalo 99, Regin 94 )
136
Exploiting Structure in QCP
ALLDIFF as Global Constraint
Matching on
Two solutions:
a Bipartite graph
All-different constraint
we can update the
domains of the column
variables
(Berge 70, Regin 94, Shaw and Walsh 98 )
137
Analogously, we can
update the domains
of the other variables
Exploiting Structure
Arc Consistency vs. All Diff
Arc Consistency
AllDiff
Solves up to order 20
Size search
space 20400
Solves up to order 40
Size search
space 401600
138
Quasigroup as Satisfiability
Two different encodings for SAT:
2D encoding (or minimal encoding);
3D encoding (or full encoding);
139
2D Encoding or Minimal Encoding
3
Variables: n
x cell i, j has color k; i, j,k =1, 2, ...,n.
ijk
x ∈ {0,1}
ijk
Each variables represents a color assigned to a cell.
Clauses:
O(n4)
• Some color must be assigned to each cell (clause of length n);
∀ (x ∨ x x )
ij ij1 ij2 ijn
• No color is repeated in the same row (sets of negative binary clauses);
∀ (¬x ∨ ¬x ) ∧ (¬x ∨ ¬x ) (¬x ∨ ¬x )
ik
i1k
i2k
i1k
i3k
i1k
ink
• No color is repeated in the same column (sets of negative binary clauses);
∀
(¬x ∨ ¬x ) ∧ (¬x ∨ ¬x ) (¬x ∨ ¬x )
jk 1 jk
2 jk
1 jk
3 jk
1 jk
njk
140
3D Encoding or Full Encoding
This encoding is based on the cubic representation of the
quasigroup: each line of the cube contains exactly one
true variable;
Variables:
Same as 2D encoding.
O(n4)
Clauses:
• Same as the 2 D encoding plus:
• Each color must appear at least once in each row;
• Each color must appear at least once in each column;
• No two colors are assigned to the sameIJCAI-01
cell;Tutorial on Phase
141
Capturing Structure Performance of SAT Solvers
State of the art backtrack and local search and complete
SAT solvers using 3D encoding are very competitive
with specialized CSP algorithms.
In contrast SAT solvers perform very poorly on 2D
encodings (SATZ or SATO);
In contrast local search solvers (Walksat) perform well
on 2D encodings;
142
SATZ on 2D encoding
(Order 20 -28)
Order 28
1,000,000
Order 20
SATZ and SATO can only solve up to order 28 when using 2D encoding;
When using 3D encoding problems of the same size take only 0 or 1
backtrack and much higher orders can be solved;
143
Walksat on 2D and 3D encoding
(Order 30-33)
1,000,000
3D order 33
2D order 33
Walksat shows an unsual pattern the 2D encodings are somewhat easier than the 3D encoding
at the peak and harder in the undereconstrained region;
144
Quasigroup - Satisfiability
Encoding the quasigroup using only
Boolean variables in clausal form using
the 3D encoding is very competitive.
Very fast solvers - SATZ, GRASP,
SATO,WALKSAT;
145
Structural features of instances provide
insights into their hardness namely:
Backbone
Inherent Structure and Balance
146
Backbone
Backbone is the shared structure of all the
solutions to a given instance.
This instance has
4 solutions:
Backbone
Total number of backbone variables: IJCAI-01
2
Tutorial on Phase
147
Phase Transition in the
Backbone
• We have observed a transition in the backbone
from a phase where the size of the backbone is
around 0% to a phase with backbone of size close
to 100%.
• The phase transition in the backbone is sudden
and it coincides with the hardest problem
instances.
(Achlioptas, Gomes, Kautz, Selman 00, Monasson et al. 99)
148
New Phase Transition in Backbone
QCP (satisfiable instances only)
% of Backbone
% Backbone
Sudden phase transition in Backbone
Computational
cost
Fraction of preassigned cells
149
Inherent Structure and Balance
150
Quasigroup Patterns and
Problems Hardness
Rectangular Pattern
Aligned Pattern
Tractable
Balanced Pattern
Very hard
(Kautz, Ruan, Achlioptas, Gomes, Selman 2001)
151
SATZ
Balanced QCP
Rectangular QCP
QCP
QWH
Aligned QCP
152
Walksat
Balanced filtered QCP
Balance QWH
QCP
QWH
aligned
rectangular
We observe the same ordering in hardness when using Walksat,
SATZ, and SATO – Balacing makes instances harder
153
Phase Transitions, Backbone,
Balance
Summary
The understanding of the structural properties of
problem instances based on notions such as
phase transitions, backbone, and balance provides
new insights into the practical complexity of many
computational tasks.
Active research area with fruitful interactions
between computer science, physics (approaches
from statistical mechanics), and mathematics
(combinatorics / random structures).
154
Outline
Randomization
Conclusions
155
Local Search
Stochastic strategies have been very successful
in the area of local search.
Simulated annealing
Genetic algorithms
Tabu Search
Gsat and variants.
Limitation: inherent incomplete nature of local
search methods.
156
Randomized Backtrack Search
Goal: explore the addition of a stochastic element to
a systematic search procedure without losing
completeness.
We introduce randomness in a backtrack search
method by randomly breaking ties in variable
and/or value selection.
Compare with standard lexicographic tiebreaking.
157
Distributions of Randomized
Backtrack Search
Key Properties:
I Erratic behavior of mean
II Distributions have “heavy tails”.
158
Erratic Behavior of Search Cost
Quasigroup Completion Problem
3500!
sample
mean
2000
Median = 1!
500
number of runs
159
Heavy-Tailed Distributions
… infinite variance … infinite mean
Introduced by Pareto in the 1920’s
--- “probabilistic curiosity.”
Mandelbrot established the use of
heavy-tailed distributions to model
real-world fractal phenomena.
Examples: stock-market, earthquakes, weather,...
160
Decay of Distributions
Standard --- Exponential Decay
e.g. Normal:
Pr[ X > x]≈ Ce − x 2, for some C > 0, x >1
Heavy-Tailed --- Power Law Decay
e.g. Pareto-Levy:
Pr[ X > x ] = C x − α , x > 0
161
Power Law Decay
Exponential Decay
Standard Distribution
(finite mean & variance)
162
How to Check for “Heavy Tails”?
Log-Log plot of tail of distribution
should be approximately linear.
Slope gives value of α
α <1
infinite mean and infinite variance
1≤α < 2
infinite variance
163
(1-F(x))(log)
Unsolved fraction
Heavy-Tailed Behavior in QCP Domain
α = 0.153
α = 0.319
18%
unsolved
α = 0.466
α <1 => Infinite mean
0.002%
unsolved
Number backtracks (log)
164
Small-World Vs. Heavy-Tailed
Behavior
Does a Small-World topology (Watts &
Strogatz) induce heavy-tail behavior?
The constraint graph of a quasigroup
exhibits a small-world topology
(Walsh 99)
165
Exploiting Heavy-Tailed Behavior
Heavy Tailed behavior has been observed in
several domains: QCP, Graph Coloring, Planning,
Scheduling, Circuit synthesis, Decoding, etc.
Consequence for algorithm design:
Use restarts or parallel / interleaved
runs to exploit the extreme variance
performance.
Restarts provably eliminate
heavy-tailed behavior.
(Gomes et al. 97, Hoos 99, Horvitz 99, Huberman, Lukose and Hogg 97, Karp et al
96, Luby et al. 93, Rish et al. 97, Wlash 99)
166
Restarts
1-F(x)
Unsolved fraction
no restarts
70%
unsolved
restart every 4 backtracks
0.001%
unsolved
250 (62 restarts)
167
Sketch of proof of elimination of
heavy tails
X − number of backtracks to solve the problem
Let’s truncate the search procedure
after m backtracks.
Probability of solving problem with truncated version:
pm = Pr[ X ≤ m]
Run the truncated procedure and restart it repeatedly.
168
Y − total number backtracks with restarts
N u m b e r o f R e s ta r ts = Y / m ~ G e o m e tr ic ( pm )




F = P r[Y > y ] ≤ (1 − pm )








Y /m




≈ c1 e − c2 y
Y - does not have Heavy Tails
169
Decoding in Communication
Systems
Voice waveform, binary digits
from a cd, output of a set of
sensors in a space probe, etc.
Telephone line, a storage
medium, a space communication
link, etc.
usually subject to NOISE
Source
Encoder
Channel
Processing prior to transmission,
e.g., insertion of redundancy to
combat the channel noise.
Decoder
Destination
Processing of the channel output with the
objective of producing at the destination
an acceptable replica of the source output.
Decoding in communication systems is NP-hard.
(Berlekamp, McEliece, and van Tilborg 1978, Barg 1998)
170
Retransmissions in Sequential
Decoding
1-F(x)
Unsolved fraction
without retransmissions
with retransmissions
Gomes et al. 2000 / 20001
171
Paramedic Crew Assignment
(Austin, Texas)
Paramedic crew assignment is the problem of assigning paramedic crews
IJCAI-01constraints.
Tutorial on Phase
from
different
stations
Gomes,
Hogg, Walsh,
Zhang to cover a given region, given several resource
172
Deterministic Search
Austin, Texas
173
Restarts
Austin, Texas
174
Results on Effectiveness of Restarts
Deterministic
Logistics Planning
Scheduling 14
108 mins.
411 sec
Scheduling 16
Scheduling 18
Circuit Synthesis 1
Circuit Synthesis 2
---(*)
---(*)
---(*)
---(*)
3
R
95 sec.
250 sec
1.4 hours
~18 hrs
165sec.
17min.
(*) not found after 2 days
175
Portfolio of Algorithms
A portfolio of algorithms is a collection of algorithms
running interleaved or on different processors.
Goal: to improve the performance of the different
algorithms in terms of:
expected runtime
“risk” (variance)
Efficient Set or Pareto set: set of portfolios that are
best in terms of expected value and risk.
(Gomes and Selman 97, Huberman, Lukose, Hogg 97 )
176
Cumulative Frequencies
Brandh & Bound for MIP
Depth-first vs. Best-bound
Optimal strategy: Best Bound
Best-Bound: Average-1400 nodes; St. Dev.- 1300
Depth-first
45%
30%
Best bound
Depth-First: Average - 18000;St. Dev. 30000
Number of nodes
177
Heavy-tailed behavior of Depth-first
178
Expected run time of portfolios
Portfolio for 6 processors
0 DF / 6 BB
3 DF / 3 BB
Efficient set
4 DF / 2 BB
6 DF / 0BB
5 DF / 1BB
Standard deviation of run time of portfolios
179
Expected run time of portfolios
Portfolio for 20 processors
0 DF / 20 BB
The optimal strategy is to run
Depth First on the 20 processors!
Optimal collective behavior emerges
from suboptimal individual behavior.
20 DF / 0 BB
Standard
deviation of run time of portfolios
180
Compute Clusters and
Distributed Agents
With the increasing popularity of
compute clusters and distributed
problem solving / agent paradigms,
portfolios of algorithms --- and flexible
computation in general --- are rapidly
expanding research areas.
(Baptista and Marques da Silva 00, Boddy & Dean 95, Bayardo 99, Davenport 00, Hogg 00,
Horvitz 96, Matsuo 00, Steinberg 00, Russell 95, Santos 99, Welman 99. Zilberstein 99)
181
Bayesian Model Structure Learning
Learning to infer predictive models from data and to identify key variables
==> restarts, cutoffs and other adaptive behavior of search algorithms.
(Horvitz, Ruan, Gomes, Kautz, Selman, Chickering 2001)
182
4XDVLJURXS 2UGHU&63
Variance in number of uncolored
cells across rows and columns
Min depth
Avg Depth
Number uncolored
cells per column
Max number of uncolored
cells across rows and columns
Green - long runs
Gray - short runs
Model accuracy 96.8% vs 48% for the marginal model
183
Analysis of different solver
features and problem features
184
Outline
Randomization
Conclusions
185
Summary
Finding optimal solution is harder than deciding
solubility
Phase transitions can be used to characterize
complex problems and their behavior
Understanding phase transitions can help to
design and develop more efficient search
algorithms
186
Summary
Phase transitions are seen in other complexity
classes
P, PSPACE, …
Structure is important to phase transition
behavior
backbones, small world topology, ....
187
Summary
The understanding of the structural properties of
problem instances based on notions such as
phase transitions, backbone, and balance provides
new insights into the practical complexity of many
computational tasks.
Active research area with fruitful interactions
between computer science, physics (approaches
from statistical mechanics), and mathematics
(combinatorics / random structures).
188
Summary
Stochastic search methods (complete and
incomplete) have been shown very effective.
Restart strategies and portfolio approaches can
lead to substantial improvements in the expected
runtime and variance, especially in the presence
of heavy-tailed phenomena.
Randomization is therefore a tool to improve
algorithmic performance and robustness.
Machine Learning techniques can be used to learn
predicitive models. IJCAI-01 Tutorial on Phase
189
Bridging the Gap
General Solution
Methods
Exploiting Structure:
Tractable Components
Transition Aware Systems
(phase transition
constrainedness
backbone resources)
Randomization
Exploits variance
to improve robustness
and performance
Real World
Problems
190

part2

Transcript

Documenti analoghi

Stata Tutorial —

Diagnosis and Identification of Discrete Event Systems using Petri

2nd Sino-Italian workshop on Graphene and related 2D Materials

CGIDEV2 Tutorial

Advances in Power Line Communications and Application

to page

5.3. PHOTON LIFETIME AND CAVITY Q