You are nicely explain situation. I just add that I lost more than 30% of my collateral because price of gwei was 500+.
I’m just wondering if the project will compensate for the addresses affected and to what extent. This move will say a lot about the quality and the direction in which the project is going.
2 Likes
that part worked, the oracle rejected 1.3 DAI price, the highest that was allowed was 1.23~
1 Like
Correct, we must gathered data about losses. But about risk management situation I think that is clear. I mentioned before that in situation where my funds are liquidated because ETH or BTC volatility - it is my fault in risk management. But in situation when I deposit Ethereum and press SAFE MAX and after 2 hours come back and see liquidation on 100% (cca. 5000% -> 6000$) because of DAI manipulation - I don’t think it’s my negligence.
I would still accept it somehow, but I didn’t have time to maneuver. As soon as I looked at the situation I started collecting ethereum on the accounts to defend myself from liquidation. It ended up that 50% of Ethereum was liquidated for peanuts, I paid a huge gas fee and after 10 minutes DAI became stablecoin again.
My opinion is that the purpose of stablecoin is to hold around the dollar however 30% of the volatility of stablecoin is totally unacceptable. That goes against the very purpose of stablecoin - why do we use them?
Imagine a situation where by manipulating DAI grows to $ 2? How many addresses would then be affected?
I was already two times liquidated but those was my faults, but this one is not.
2 Likes
Apologies about not being clear; I meant that managing the risk of these liquidations happening again due to this specific oracle issue should be prioritized, I was not making a comment on anyone’s financial risk management.
3 Likes
No bad feelings 
,
I admit that sometimes I have bad risk management, I commented on your post because everything you said makes sense. I followed up on risk management to present the situation I found myself in - helpless as on a roulette table.
2 Likes
I’d argue the opposite side of this point. If DAI were to ever trade at a true 30% premium for any significant length of time (longer than a few hours), this would most likely result in emergency shutdown, and all outstanding DAI being settled for $1 worth of collateral.
I don’t think it makes sense to allow for DAIUSD oracle prices above around ~$1.10 because the value for cDAI holders that Compound would be protecting with liquidations is basically fictitious.
3 Likes
I remember this in documentation too. I assumed that the TWAP used an interval window that was a little longer- if the price registered at 1.23 like @blck noted it seems like the time window over which average price is calculated is a bit too short. Might be good to have it a bit longer so that the window covers a peak volume time and/or integrate a volume weighted element to the feed [ie do a VWAP over the extended time window, or calculate the average price of the last $1mm in transacted].
1 Like
I’m generally more in the dakeshi camp on this one, but I see your point. Given that one might be okay with neglecting the trade price for a few hours if it seems “nonsensical” [since we might be willing to cap the rate at 1.1 given that price really shouldn’t be above that level for more than a few hours], perhaps a more straightforward approach might be to use a longer time interval for the TWAP calculation. This way you allow for the vol/stability of the trade price to get baked in as dakeshi advocates, plus you get some stickiness in price to help it weather some odd short term moves that might take some time to get arbed away through minting etc (may take especially long if network is congested), which is what the cap sort of gets at.
2 Likes
To fix this problem at its core, Compound needs to integrate Chainlink Price Feeds. I am writing this as a daily DeFi user and as someone who only wants the best for the DeFi ecosystem as a whole, especially as the value secured rises. The false liquidation of ~$90M in user funds today was a serious issue that was directly caused by Compound’s centralized oracle solution which pulls market data from only a single exchange, Coinbase, with Uniswap TWAP used as a backstop. Compound’s price feeds provide data that only reflects a small subset of the total crypto trading market and fundamentally cannot provide sufficient market coverage. This in turn lowers the cost of market manipulation and exposes the protocol to inaccurate pricing from large trades.
Specifically, Coinbase has an extensive history of downtime and flash crashes, so I am surprised this was not immediately seen during development as being a huge single point of failure. Using Uniswap TWAP as a backstop is better than no backstop in this situation, but it introduces a false sense of security as it too can trivially be manipulated (as we saw during this event). This lack of market coverage allowed a malicious actor to manipulate just two exchanges to skew the price data delivered to the Compound protocol and falsely liquidate users and yield farmers using DAI as debt or collateral. The core issues of price feeds without market coverage are covered extensively in this blog post here which provides context about the importance of data quality for oracles.
Coinbase was the only major exchange that experienced such a drastic price deviation, other major exchanges were unaffected.
However, none of this information I mention above is new, as I have previously pointed out the numerous and specific vulnerabilities in the design of Compound’s oracle that were not and still have yet to be fixed. Here is a tweet thread I wrote on July 21st 2020 on my concerns regarding the Compound oracle and the likelihood of Coinbase experiencing market manipulation/flash crashes, the ability to manipulate Uniswap TWAP, and why taking a simple median across pre-selected exchanges does not solve the issue adequately either. Compound’s price oracles are still highly vulnerable to these issues as we speak, leaving over $3B in user deposited funds at risk of further catastrophic losses, and needs to be fixed immediately. Compound’s price oracle simply does not provide adequate market coverage as it exists today. Moreover, because it requires exchanges to change their API infrastructure to provide signed data that is compatible with Ethereum, the Compound oracle will continue to be inherently limited in the amount of market coverage it can ever achieve.
Chainlink Price Feeds provide an immediate solution to this problem, allowing the Compound protocol to fully mitigate these oracle related issues going forward. Aave, another decentralized money market on Ethereum experienced no price oracle issues during this event or any false liquidations. There is a very simple reason for this; instead of rolling their own oracle, exposing them to wide range of nuanced attack vectors, they simply integrated Chainlink oracles, which has successfully provided Aave users with the true market wide price of both DAI and every other asset on the platform since launch, as well as during this Coinbase/Uniswap outlier flash crash. I implore you to consider the following sections as I describe how Chainlink is resilient to these attack vectors.
Chainlink’s Decentralized Price Feeds are highly accurate and resistant to exchange distortions because they provide full market coverage by using multiple layers of aggregation that smooth outliers and prevent manipulated data from being delivered to smart contracts. This ensures market manipulation on a select few exchanges have no effect on the final data point generated and delivered to contracts. Specifically, Chainlink has three levels of aggregation to prevent the exact issues Compound’s price oracles experienced today.
- Firstly, Chainlink uses professional data providers (CoinGecko, BraveNewCoin, Amberdata, Kaiko, CryptoCompare, Alpha Vantage, CoinApi, CoinPaprika, CryptoAPIs, and more) who whose entire business model revolves around generating high quality data using refined aggregation methodologies. These data providers produce reference prices for cryptocurrencies that reflect the market-wide price by tracking hundreds of exchanges (both on-chain DEXs and off-chain CEXs), taking into account volume, liquidity, time, and other shifting differences across exchanges, preventing any single source of truth.
- Secondly, there are the security reviewed Chainlink node operators (T-Systems, LinkPool, Certus.One, Stake.fish, Chainlayer, Chorus,one, SNZ, Huobi, and dozens more) operated by professional DevOps and blockchain infrastructure teams who aggregate price data from multiple data aggregators and take the median off-chain before delivering the data point on-chain, preventing any single source of truth. These Chainlink nodes are paid for their services in LINK, not only covering their gas costs, ensuring timely and incentivized updates, but providing a source of profit. This creates crypto-economic security by creating a large opportunity cost for malicious activity. Additionally, multiple data providers already operate their own Chainlink oracle node and provide cryptographically signed data.
- Thirdly, there are the Chainlink oracle networks (feeds.chain.link) which are on-chain reference contracts that aggregate data from multiple node operators, again preventing any single source of truth. Each Price Feed is updated based on a threshold deviation and a heartbeat frequency, ensuring fresh data that follows market volatility is always available to contracts. These Price Feeds are a shared public good funded by many DeFi projects and already secure over $4B in user funds.
What I am proposing here is quite simple. By integrating Chainlink Price Feeds as the primary oracle solution for the Compound protocol, these market coverage issues simply disappear and users can be assured they will not be falsely liquidated (just as Aave can today). Chainlink already supports all of the price feeds the Compound protocol needs on mainnet and integration would be straight-forward, only requiring a few lines of code (docs.chain.link). Additionally, Chainlink Price Feeds can also be used in replacement of Uniswap as the backstop, providing a much more tamper-resistant solution, though being the primary oracle is ideal as it would completely stop these exploits from occurring and ensure there is no period without accurate data. I am writing this as a concerned DeFi user who does not want to see more user funds falsely liquidated due to entirely preventable oracle issues. We are all in this together and I believe that the DeFi community can come together to ensure all protocols are using oracle solutions that are sufficiently secure for the value they secure.
Please take what I say with consideration as the value locked in DeFi continues to grow in orders of magnitude. By fixing the issue at its source now, Compound development and governance can focus on and innovate around what assets should be listed and the risk parameters, rather than worrying about how to refund users in the wake of another price oracle exploit.
I have created a discussion for this proposal here.
10 Likes
Coinbase looked at the data from many different angles and concluded that not only were there no price manipulation alerts in our trade surveillance software, but we did not find any evidence of collusion or single actors that pushed the price higher. We believe our books properly operated according to the availably liquidity at the time which, in stablecoin markets, may become thin as price moves away due to the collateralized nature of these markets. And, Coinbase Price Oracle accurately reported data throughout the DAI price increase.
Pete Elkins
Head of Trade Surveillance and Market Health
Coinbase
5 Likes
Thank you, for the report, Pete. The thing is that even if we take as a given that price, reported by Coinbase oracle was accurate for the actual market conditions on particular Coinbase market and wasn’t a result of error or manipulation, it wasn’t relevant to actual price of DAI, as other market’s, in particular, Uniswap have not experienced and reported such price. Since it’s critical for Compound to have accurate price at every time, event proved, that Coinbase Price Oracle is not enough by itself to achive that goal. That doesn’t mean that price reported by Coinbase isn’t important data source, but it looks like just by itself, it’s not going to do the trick.
3 Likes
Thank you for chiming in. While I find your discoveries hard to believe, I appreciate you coming here to share your perspective.
First, it is widely understood that the market price was $1.03 during this incident. There were millions of DAI available for sale at this price across nearly every exchange in these moments. Can you please comment on how and why the Coinbase Oracle failed to reflect true market value despite three promised layers of protection against off market data?
Specifically I’d like your comment on point #2, off-chain filtering. Everyone knows DAI should trade near $1, and given that it was clear that in this moment, there was a liquidity crisis on Coinbase and Coinbase alone, why was $1.30 allowed to be printed on the oracle? Especially since it was only sustained for a few minutes. The oracle should be smarter.
This is completely unacceptable for Coinbase to be so cavalier about reporting local market prices on an narrow illiquid slice of the market when everyone knows the real purpose of the oracle is to report true market value. Your blog post says as much. Again it states “it is important to address various scenarios in which a data point to be signed does not reflect an actual market price of an asset”.
What sanity checks did the oracle perform? Did it check to see if Coinbase was off-market relative to other exchanges? In this case, every other exchange. I’d like to know how this oracle failure happened. I’d also like a full disclosure of how many accounts participated in driving the price of DAI to $1.30 and keeping it there. Please disclose whether there was anything suspicious or any patterns amongst these accounts.
5 Likes
Hi, Pete.
Thanks for this.
Was there anything out of the ordinary with regard to the DAI-USDC designated market makers? Specifically:
- Were the market markets operating normally?
- What kept them from bringing back down the price by buying DAI on the Coinbase DAI-USD book and selling on DAI-USDC?
- The price dislocation against the wider market lasted for about an hour, so I think market makers would have had time to source liquidity even from other exchanges. Why didn’t they?
- Are you planing on increasing market makers on DAI-USDC to prevent other such liquidity crunches? I analyzed the relevant trades for this event, and buying about 360,000 DAI with a total cost of about 21K USD (paying above $1) is what was required to move the price to $1.3.
5 Likes
I’m a long time Compound user, but not integrating Chainlink at this point would be a mistake. It’s been mentioned in this thread many different ways already, but if you aren’t getting full market coverage, us users are taking on unnecessary risk. This recent problem with the DAI price being manipulated is proof of that. What would be a good reason NOT to integrate Chainlink price feeds?
8 Likes
MakerDAO offered liquidity against USDC, TUSD and PAX with up to ~99% LTV (100x leverage) and no liquidations, so it seems like it should be impossible for the price to diverge this much if deposits and withdrawals were operating normally.
Can you comment on the existence of any deposit/withdrawal issues during the time period covered? If there were issues, were they caused by technical faults or did Coinbase restrict transfers purposefully?
Thank you for your consideration.
9 Likes
Esteemed kybx86 has created a thread “Compensation Proposal: Distribute COMP to Affected Users in the DAI Liquidations” - I suggest everybody to move there, to that thread for the discussion of the specifics.
1 Like
Hi @cryptoguy123. I’ve been following your work, and I agree that Coinbase’s filtering was poor judgment and candidly frustrating. I was also one of the affected users and I’ve been working with the community on compensation. Not sure if you’ve seen it, but I deployed a proposal to compensate users. It needs the community’s votes to pass. You can view the full details under this forum post: Compensation Proposal: Distribute COMP to Affected Users in the DAI Liquidations
2 Likes
Ideation for a new proposal that addresses objections with the initial has been occurring in the thread for the previous proposal Compensation Proposal: Distribute COMP to Affected Users in the DAI Liquidations
1 Like
1- My thought is that it’s not the role of governance token to compensate for liquidations or failed liquidations
2 - In case there is a reserve build( the DAI reserve) it might be used after a vote to compensate for that day, even if it is not in this case a failed liquidation, but it should be voted whether or not the insurance fund should cover it
3- in case the vote would be to cover it, the coverage shouldnt go higher than the insurance fund taken at the time of the event, should the loss be higher.
insurance fund(dai reserve) is there to insure for failed liquidation
comp token was there to distribute a governance power to the right persons, not here to be used as an insurance fund, if this changes, it must be decided in the future and not related to a specific event
here is my opinion
2 Likes
I’d say I agree that deployment of an insurance fund makes perfect sense for covering platform failure / false liquidations over distributing governance tokens. I’d prefer the former over the latter.
(That being said my #1 preference would be returning the assets seized in false liquidation but I do not know the feasibility of that)
2 Likes